Blame doc/chapters/largerpost.inc

Packit 875988
The previous chapter introduced a way to upload data to the server, but the developed example program 
Packit 875988
has some shortcomings, such as not being able to handle larger chunks of data. In this chapter, we
Packit 875988
are going to discuss a more advanced server program that allows clients to upload a file in order to 
Packit 875988
have it stored on the server's filesystem. The server shall also watch and limit the number of
Packit 875988
clients concurrently uploading, responding with a proper busy message if necessary.
Packit 875988
Packit 875988
Packit 875988
@heading Prepared answers
Packit 875988
We choose to operate the server with the @code{SELECT_INTERNALLY} method. This makes it easier to 
Packit 875988
synchronize the global states at the cost of possible delays for other connections if the processing
Packit 875988
of a request is too slow. One of these variables that needs to be shared for all connections is the
Packit 875988
total number of clients that are uploading.
Packit 875988
Packit 875988
@verbatim
Packit 875988
#define MAXCLIENTS      2
Packit 875988
static unsigned int    nr_of_uploading_clients = 0;
Packit 875988
@end verbatim
Packit 875988
@noindent
Packit 875988
Packit 875988
If there are too many clients uploading, we want the server to respond to all requests with a busy
Packit 875988
message.
Packit 875988
@verbatim
Packit 875988
const char* busypage = 
Packit 875988
  "<html><body>This server is busy, please try again later.</body></html>";
Packit 875988
@end verbatim
Packit 875988
@noindent
Packit 875988
Packit 875988
Otherwise, the server will send a @emph{form} that informs the user of the current number of uploading clients,   
Packit 875988
and ask her to pick a file on her local filesystem which is to be uploaded. 
Packit 875988
@verbatim
Packit 875988
const char* askpage = "<html><body>\n\
Packit 875988
                       Upload a file, please!
\n\
Packit 875988
                       There are %u clients uploading at the moment.
\n\
Packit 875988
                       
Packit 875988
                         enctype=\"multipart/form-data\">\n\
Packit 875988
                       <input name=\"file\" type=\"file\">\n\
Packit 875988
                       <input type=\"submit\" value=\" Send \"></form>\n\
Packit 875988
                       </body></html>";
Packit 875988
@end verbatim
Packit 875988
@noindent
Packit 875988
Packit 875988
If the upload has succeeded, the server will respond with a message saying so.
Packit 875988
@verbatim
Packit 875988
const char* completepage = "<html><body>The upload has been completed.</body></html>";
Packit 875988
@end verbatim
Packit 875988
@noindent
Packit 875988
Packit 875988
We want the server to report internal errors, such as memory shortage or file access problems,
Packit 875988
adequately. 
Packit 875988
@verbatim
Packit 875988
const char* servererrorpage 
Packit 875988
  = "<html><body>An internal server error has occured.</body></html>";
Packit 875988
const char* fileexistspage
Packit 875988
  = "<html><body>This file already exists.</body></html>";
Packit 875988
@end verbatim
Packit 875988
@noindent
Packit 875988
Packit 875988
It would be tolerable to send all these responses undifferentiated with a @code{200 HTTP_OK} 
Packit 875988
status code but in order to improve the @code{HTTP} conformance of our server a bit, we extend the 
Packit 875988
@code{send_page} function so that it accepts individual status codes. 
Packit 875988
Packit 875988
@verbatim
Packit 875988
static int 
Packit 875988
send_page (struct MHD_Connection *connection, 
Packit 875988
	   const char* page, int status_code)
Packit 875988
{
Packit 875988
  int ret;
Packit 875988
  struct MHD_Response *response;
Packit 875988
  
Packit 875988
  response = MHD_create_response_from_buffer (strlen (page), (void*) page, 
Packit 875988
  	     				      MHD_RESPMEM_MUST_COPY);
Packit 875988
  if (!response) return MHD_NO;
Packit 875988
 
Packit 875988
  ret = MHD_queue_response (connection, status_code, response);
Packit 875988
  MHD_destroy_response (response);
Packit 875988
Packit 875988
  return ret;
Packit 875988
}
Packit 875988
@end verbatim
Packit 875988
@noindent
Packit 875988
Packit 875988
Note how we ask @emph{MHD} to make its own copy of the message data. The reason behind this will
Packit 875988
become clear later.
Packit 875988
Packit 875988
Packit 875988
@heading Connection cycle
Packit 875988
The decision whether the server is busy or not is made right at the beginning of the connection. To 
Packit 875988
do that at this stage is especially important for @emph{POST} requests because if no response is 
Packit 875988
queued at this point, and @code{MHD_YES} returned, @emph{MHD} will not sent any queued messages until
Packit 875988
a postprocessor has been created and the post iterator is called at least once.
Packit 875988
Packit 875988
@verbatim
Packit 875988
static int 
Packit 875988
answer_to_connection (void *cls, struct MHD_Connection *connection, 
Packit 875988
		      const char *url, 
Packit 875988
                      const char *method, const char *version, 
Packit 875988
		      const char *upload_data, 
Packit 875988
                      size_t *upload_data_size, void **con_cls)
Packit 875988
{
Packit 875988
  if (NULL == *con_cls) 
Packit 875988
    {
Packit 875988
      struct connection_info_struct *con_info;
Packit 875988
Packit 875988
      if (nr_of_uploading_clients >= MAXCLIENTS) 
Packit 875988
        return send_page(connection, busypage, MHD_HTTP_SERVICE_UNAVAILABLE);
Packit 875988
@end verbatim
Packit 875988
@noindent
Packit 875988
Packit 875988
If the server is not busy, the @code{connection_info} structure is initialized as usual, with 
Packit 875988
the addition of a filepointer for each connection.
Packit 875988
Packit 875988
@verbatim
Packit 875988
      con_info = malloc (sizeof (struct connection_info_struct));
Packit 875988
      if (NULL == con_info) return MHD_NO;
Packit 875988
      con_info->fp = 0;
Packit 875988
Packit 875988
      if (0 == strcmp (method, "POST")) 
Packit 875988
        {  
Packit 875988
          ...
Packit 875988
        } 
Packit 875988
      else con_info->connectiontype = GET;
Packit 875988
Packit 875988
      *con_cls = (void*) con_info;
Packit 875988
 
Packit 875988
      return MHD_YES;
Packit 875988
    }
Packit 875988
@end verbatim
Packit 875988
@noindent
Packit 875988
Packit 875988
For @emph{POST} requests, the postprocessor is created and we register a new uploading client. From 
Packit 875988
this point on, there are many possible places for errors to occur that make it necessary to interrupt
Packit 875988
the uploading process. We need a means of having the proper response message ready at all times. 
Packit 875988
Therefore, the @code{connection_info} structure is extended to hold the most current response
Packit 875988
message so that whenever a response is sent, the client will get the most informative message. Here,
Packit 875988
the structure is initialized to "no error".
Packit 875988
@verbatim
Packit 875988
      if (0 == strcmp (method, "POST")) 
Packit 875988
        {  
Packit 875988
          con_info->postprocessor 
Packit 875988
	    = MHD_create_post_processor (connection, POSTBUFFERSIZE, 
Packit 875988
                                         iterate_post, (void*) con_info);   
Packit 875988
Packit 875988
          if (NULL == con_info->postprocessor) 
Packit 875988
            {
Packit 875988
              free (con_info); 
Packit 875988
              return MHD_NO;
Packit 875988
            }
Packit 875988
Packit 875988
          nr_of_uploading_clients++;
Packit 875988
          
Packit 875988
          con_info->connectiontype = POST;
Packit 875988
          con_info->answercode = MHD_HTTP_OK;
Packit 875988
          con_info->answerstring = completepage;
Packit 875988
        } 
Packit 875988
      else con_info->connectiontype = GET;
Packit 875988
@end verbatim
Packit 875988
@noindent
Packit 875988
Packit 875988
If the connection handler is called for the second time, @emph{GET} requests will be answered with
Packit 875988
the @emph{form}. We can keep the buffer under function scope, because we asked @emph{MHD} to make its
Packit 875988
own copy of it for as long as it is needed.
Packit 875988
@verbatim
Packit 875988
  if (0 == strcmp (method, "GET")) 
Packit 875988
    {
Packit 875988
      int ret;
Packit 875988
      char buffer[1024];
Packit 875988
        
Packit 875988
      sprintf (buffer, askpage, nr_of_uploading_clients);
Packit 875988
      return send_page (connection, buffer, MHD_HTTP_OK);     
Packit 875988
    } 
Packit 875988
@end verbatim
Packit 875988
@noindent
Packit 875988
Packit 875988
Packit 875988
The rest of the @code{answer_to_connection} function is very similar to the @code{simplepost.c}
Packit 875988
example, except the more flexible content of the responses. The @emph{POST} data is processed until
Packit 875988
there is none left and the execution falls through to return an error page if the connection
Packit 875988
constituted no expected request method.
Packit 875988
@verbatim
Packit 875988
  if (0 == strcmp (method, "POST")) 
Packit 875988
    {
Packit 875988
      struct connection_info_struct *con_info = *con_cls;
Packit 875988
       
Packit 875988
      if (0 != *upload_data_size) 
Packit 875988
        { 
Packit 875988
          MHD_post_process (con_info->postprocessor,
Packit 875988
	                    upload_data, *upload_data_size);
Packit 875988
          *upload_data_size = 0;
Packit 875988
          
Packit 875988
          return MHD_YES;
Packit 875988
        } 
Packit 875988
      else 
Packit 875988
        return send_page (connection, con_info->answerstring, 
Packit 875988
	       		  con_info->answercode);
Packit 875988
    } 
Packit 875988
Packit 875988
  return send_page(connection, errorpage, MHD_HTTP_BAD_REQUEST);
Packit 875988
}
Packit 875988
@end verbatim
Packit 875988
@noindent
Packit 875988
Packit 875988
Packit 875988
@heading Storing to data
Packit 875988
Unlike the @code{simplepost.c} example, here it is to be expected that post iterator will be called
Packit 875988
several times now. This means that for any given connection (there might be several concurrent of them)
Packit 875988
the posted data has to be written to the correct file. That is why we store a file handle in every
Packit 875988
@code{connection_info}, so that the it is preserved between successive iterations.
Packit 875988
@verbatim
Packit 875988
static int 
Packit 875988
iterate_post (void *coninfo_cls, enum MHD_ValueKind kind, 
Packit 875988
	      const char *key,
Packit 875988
	      const char *filename, const char *content_type,
Packit 875988
              const char *transfer_encoding, const char *data, 
Packit 875988
	      uint64_t off, size_t size)
Packit 875988
{
Packit 875988
  struct connection_info_struct *con_info = coninfo_cls;
Packit 875988
@end verbatim
Packit 875988
@noindent
Packit 875988
Packit 875988
Because the following actions depend heavily on correct file processing, which might be error prone,
Packit 875988
we default to reporting internal errors in case anything will go wrong.
Packit 875988
Packit 875988
@verbatim
Packit 875988
con_info->answerstring = servererrorpage;
Packit 875988
con_info->answercode = MHD_HTTP_INTERNAL_SERVER_ERROR;
Packit 875988
@end verbatim
Packit 875988
@noindent
Packit 875988
Packit 875988
In the "askpage" @emph{form}, we told the client to label its post data with the "file" key. Anything else
Packit 875988
would be an error.
Packit 875988
Packit 875988
@verbatim
Packit 875988
  if (0 != strcmp (key, "file")) return MHD_NO;
Packit 875988
@end verbatim
Packit 875988
@noindent
Packit 875988
Packit 875988
If the iterator is called for the first time, no file will have been opened yet. The @code{filename}
Packit 875988
string contains the name of the file (without any paths) the user selected on his system. We want to
Packit 875988
take this as the name the file will be stored on the server and make sure no file of that name exists
Packit 875988
(or is being uploaded) before we create one (note that the code below technically contains a
Packit 875988
race between the two "fopen" calls, but we will overlook this for portability sake).
Packit 875988
@verbatim
Packit 875988
  if (!con_info->fp)
Packit 875988
    {
Packit 875988
      if (NULL != (fp = fopen (filename, "rb")) )
Packit 875988
        {
Packit 875988
          fclose (fp);
Packit 875988
          con_info->answerstring = fileexistspage;
Packit 875988
          con_info->answercode = MHD_HTTP_FORBIDDEN;
Packit 875988
          return MHD_NO;
Packit 875988
        }
Packit 875988
      
Packit 875988
      con_info->fp = fopen (filename, "ab");
Packit 875988
      if (!con_info->fp) return MHD_NO;    
Packit 875988
    }
Packit 875988
@end verbatim
Packit 875988
@noindent
Packit 875988
Packit 875988
Packit 875988
Occasionally, the iterator function will be called even when there are 0 new bytes to process. The 
Packit 875988
server only needs to write data to the file if there is some.
Packit 875988
@verbatim
Packit 875988
if (size > 0) 
Packit 875988
    {  
Packit 875988
      if (!fwrite (data, size, sizeof(char), con_info->fp))
Packit 875988
        return MHD_NO;
Packit 875988
    }
Packit 875988
@end verbatim
Packit 875988
@noindent
Packit 875988
Packit 875988
If this point has been reached, everything worked well for this iteration and the response can
Packit 875988
be set to success again. If the upload has finished, this iterator function will not be called again.
Packit 875988
@verbatim
Packit 875988
  con_info->answerstring = completepage;
Packit 875988
  con_info->answercode = MHD_HTTP_OK;
Packit 875988
Packit 875988
  return MHD_YES;
Packit 875988
}
Packit 875988
@end verbatim
Packit 875988
@noindent
Packit 875988
Packit 875988
Packit 875988
The new client was registered when the postprocessor was created. Likewise, we unregister the client
Packit 875988
on destroying the postprocessor when the request is completed.
Packit 875988
@verbatim
Packit 875988
void request_completed (void *cls, struct MHD_Connection *connection, 
Packit 875988
     		        void **con_cls,
Packit 875988
                        enum MHD_RequestTerminationCode toe)
Packit 875988
{
Packit 875988
  struct connection_info_struct *con_info = *con_cls;
Packit 875988
Packit 875988
  if (NULL == con_info) return;
Packit 875988
Packit 875988
  if (con_info->connectiontype == POST)
Packit 875988
    {
Packit 875988
      if (NULL != con_info->postprocessor) 
Packit 875988
        {
Packit 875988
          MHD_destroy_post_processor (con_info->postprocessor); 
Packit 875988
          nr_of_uploading_clients--;
Packit 875988
        }
Packit 875988
Packit 875988
      if (con_info->fp) fclose (con_info->fp); 
Packit 875988
    }
Packit 875988
Packit 875988
  free (con_info);
Packit 875988
  *con_cls = NULL;      
Packit 875988
}
Packit 875988
@end verbatim
Packit 875988
@noindent
Packit 875988
Packit 875988
Packit 875988
This is essentially the whole example @code{largepost.c}.
Packit 875988
Packit 875988
Packit 875988
@heading Remarks
Packit 875988
Now that the clients are able to create files on the server, security aspects are becoming even more
Packit 875988
important than before. Aside from proper client authentication, the server should always make sure
Packit 875988
explicitly that no files will be created outside of a dedicated upload directory.  In particular,
Packit 875988
filenames must be checked to not contain strings like "../".