code.delx.au - gnu-emacs-elpa/blob - packages/web-server/NOTES

   1                                                            -*- org -*-
   2 #+Title: Notes and Tasks
   3 #+HTML_HEAD: <style>pre{background:#232323; color:#E6E1DC;} table{margin:auto; width:50%;} @media(min-width:800px){div#content{max-width:800px; padding:2em; margin:auto;}}</style>
   4 #+Options: ^:{}
   5
   6 * Notes
   7 * Tasks [19/22]
   8 ** DONE pass all tests on Windows [2/2]
   9 Currently two failing tests.
  10
  11 - [X] ws/simple-post returns "you said nil" instead of "you said foo"
  12
  13 - [X] ws/in-directory-p is failing because it assumes "/tmp" which
  14   doesn't work on windows
  15
  16 ** DONE Content and Transfer encodings
  17 - Content and Transfer encoding values
  18   http://www.iana.org/assignments/http-parameters/http-parameters.xhtml
  19 - http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.6
  20 - http://en.wikipedia.org/wiki/Chunked_transfer_encoding
  21 - http://tools.ietf.org/html/rfc1945#section-7.2
  22 - http://tools.ietf.org/rfc/rfc1945.txt
  23
  24 Some issue as to whether compression is better done as a "Content
  25 Encoding" which actually changes the content, or as a "Transfer
  26 Encoding", which doesn't change the content, just the messages.
  27
  28 The latter seems preferable, but possibly less widely supported.  See
  29 http://stackoverflow.com/questions/11641923/transfer-encoding-gzip-vs-content-encoding-gzip.
  30
  31 - content-coding
  32   - compress :: Unix =compress= program (rfc2616)
  33   - deflate :: =zlib= (see http://www.iana.org/go/rfc1950) format with
  34        =defalte= compression (rfc2616)
  35   - exi :: W3c efficient XML (see http://www.w3.org/TR/exi/)
  36   - gzip :: GNU zip (rfc2616)
  37   - identity :: does nothing
  38   - pack200-zip :: specific to Java archives (see
  39        http://www.jcp.org/en/jsr/detail?id=200)
  40 - transfer-coding
  41   - chunked :: (rfc2616)
  42   - compress :: same as above
  43   - deflate :: same as above
  44   - gzip :: same as above
  45 - tail-header
  46   - Content-MD5 :: Base64 encoded binary MD5 sum of content
  47
  48 Maybe we can set the coding system of the process with
  49 =define-coding-system=, specifically using the =:pre-write-conversion=
  50 flag to e.g., gzip or chunkify the contents.
  51
  52 ** DONE web sockets
  53 - http://en.wikipedia.org/wiki/WebSocket
  54 - http://tools.ietf.org/html/rfc6455
  55
  56 ** more examples [4/4]
  57 *** DONE Org-mode agenda
  58 Already exists as part of org-ehtml.
  59 file:examples/011-org-agenda.el
  60
  61 *** DONE display the current buffer
  62 Idea stolen from elnode.
  63 file:examples/010-current-buffer.el
  64
  65 *** DONE browse the BBDB
  66 file:examples/012-search-bbdb.el
  67 *** DONE org-mode export server
  68 1. upload a file
  69 2. supply an export type
  70 3. return the exported version of the file
  71
  72 ** DONE handle large files
  73 When large files arrive quickly, the filter functions are called while
  74 they are still running on the previous chunk, this leads to nasty race
  75 conditions for the state of the request object.
  76
  77 Either introduce some check to wait on new input if input is currently
  78 being parsed, or wait until all input has arrived before doing any
  79 parsing.
  80
  81 Now using an =active= field on request objects to avoid race
  82 conditions when new header text is received while the parsing function
  83 is still active.
  84
  85 ** TODO robustness to bad requests [0/2]
  86 Low priority, just [[*running%20behind%20a%20proxy][run behind a proxy]].
  87
  88 *** TODO request timeout
  89 *** TODO maximum request size
  90 ** DONE authentication [2/2]
  91    - State "HOLD"       from "TODO"       [2014-02-10 Mon 19:06] \\
  92      digest may not be worth it, just run Basic over HTTPS
  93 *** DONE Basic
  94 http://en.wikipedia.org/wiki/Basic_access_authentication
  95
  96 *** CANCELED Digest
  97 http://en.wikipedia.org/wiki/Digest_access_authentication
  98
  99 If this is implemented, it would be good to implement some safeguards
 100 against common attacks.
 101
 102 #+begin_quote
 103 - Server nonce is allowed to contain timestamps. Therefore the server
 104   may inspect nonce attributes submitted by clients, to prevent replay
 105   attacks.
 106 - Server is also allowed to maintain a list of recently issued or used
 107   server nonce values to prevent reuse.
 108 #+end_quote
 109
 110 ** incremental handler calls
 111 not sure if the extra performance is worth the added complexity
 112
 113 Before the header is fully parsed, call any potential handlers.
 114 Include a field in the request object to indicate that the request
 115 isn't finished being received so handlers can return and wait to be
 116 called again.
 117
 118 Also, put a catch in the filter function and allow the =headers=
 119 function on the request object to throw to said catch aborting the
 120 handler and waiting for the rest of the input.
 121
 122 ** DONE Documentation [6/6]
 123 - [X] introduction
 124 - [X] handlers
 125 - [X] request headers
 126 - [X] usage examples
 127 - [X] list of functions
 128
 129 Notes to touch upon
 130 - [X] how to set content type
 131
 132 ** DONE Handle POST requests
 133 1. read standard for POST data
 134 2. parse multi-line headers with boundaries
 135
 136 For now keep this all incremental and in ws-filter.
 137
 138 ** DONE Makefile
 139 - byte-compile
 140 - package
 141 - test
 142 - benchmark
 143 ** DONE catch errors and return an error code
 144 include an easy error handler like the 404 handler
 145
 146 ** DONE better parsing of multipart form blocks
 147 parse more than just the content-type headers.
 148
 149 ** DONE non-multipart form data
 150 e.g., parameter strings
 151
 152 ** DONE some more convenience functionality [6/6]
 153 - [X] strip and parse URL query string
 154 - [X] parse urlencoded post data
 155 - [X] think about defaulting to (name . content) for form elements
 156 - [X] maybe don't require a non-nil return to cancel the connection,
 157       instead only keep open if =:keep-alive= is returned
 158 - [X] function to send a file (with mime handling)
 159 - [X] send a 404 with some default text
 160
 161 ** CANCELED Lazy header processing
 162    - State "CANCELED"   from "TODO"       [2013-12-25 Wed 12:21] \\
 163      premature optimization
 164 Use lazy sequence functions for header a-list to avoid parsing all
 165 headers.  For regexp matchers should stop when matched header is
 166 encountered (often the first one when :GET), For function matchers
 167 provide lazy version of assoc.
 168
 169 Also, there is the issue of how a lazy request for more parameters
 170 should act before all incoming text has been received.  Emacs does not
 171 provide a light-weight mechanism for a function to wait for incoming
 172 process text without something gross like the =(sit-for 0.1)= used in
 173 the test suite.
 174
 175 ** use gnutls for https
 176 low priority -- just [[*running%20behind%20an%20https%20proxy][run behind an https proxy]].
 177
 178 This will be a pain, and will require expanding [[info:emacs-gnutls]] to
 179 add support for starting server processes, currently only client
 180 processes are supported.
 181 ** screen cast?
 182 - http://en.wikipedia.org/wiki/XVidCap
 183 - https://aur.archlinux.org/packages/xvidcap/
 184
 185 * Tutorials
 186 The following tutorials walk through common usage scenarios including
 187 installing the Emacs web-server and running it behind a proxy.
 188 Install the Emacs web-server and run =(info "web-server")= to browse
 189 the full manual within Emacs, or view the HTML version at
 190 [[http://eschulte.github.io/emacs-web-server/][emacs-web-server]].
 191
 192 ** Installation and running a server
 193 Most easily installable through the GNU ELPA, run =M-x
 194 package-list-packages= select =web-server= and install.  Alternately,
 195 install from the git repository at
 196 https://github.com/eschulte/emacs-web-server and update your the load.
 197
 198 1. Ensure that you have Emacs version 24 or greater installed.
 199
 200    #+begin_src sh :results scalar
 201      emacs --version
 202    #+end_src
 203
 204    : GNU Emacs 24.3.1
 205    : Copyright (C) 2013 Free Software Foundation, Inc.
 206    : GNU Emacs comes with ABSOLUTELY NO WARRANTY.
 207    : You may redistribute copies of Emacs
 208    : under the terms of the GNU General Public License.
 209    : For more information about these matters, see the file named COPYING.
 210
 211 2. Download and unpack the zip archive of the Emacs web-server code
 212    from [[https://github.com/eschulte/emacs-web-server/archive/master.zip][emacs-web-server-master.zip]] or clone the source code
 213    repository with [[http://git-scm.com/][git]].
 214
 215    #+begin_src sh
 216      git clone https://github.com/eschulte/emacs-web-server.git
 217    #+end_src
 218
 219 3. Move into the root of the =emacs-web-server/= directory and
 220    optionally run =make= to compile the web-server code, and run =make
 221    check= to test your web-server install.
 222
 223    #+begin_src sh
 224      make
 225      make check
 226    #+end_src
 227
 228 4. From the root of the =emacs-web-server/= directory, start an
 229    instance of Emacs with web-server loaded.
 230
 231    #+begin_src sh
 232      emacs -Q -L . -l web-server
 233    #+end_src
 234
 235    Alternately, from an already running Emacs instance, add this
 236    directory to the load path and load the web server with the
 237    following.
 238
 239    #+begin_src emacs-lisp
 240      (add-to-list 'load-path "path/to/emacs-web-server")
 241      (require 'web-server)
 242    #+end_src
 243
 244 5. Evaluate the following code in =*scratch*= buffer of this Emacs
 245    instance.
 246
 247    #+begin_src emacs-lisp
 248      (ws-start
 249       (lambda (request)
 250         (with-slots (process headers) request
 251           (ws-response-header process 200 '("Content-type" . "text/plain"))
 252           (process-send-string process "hello world")))
 253       9000)
 254    #+end_src
 255
 256 6. Browse to http://localhost:9000 to see that the web-server is
 257    running.
 258
 259 7. Read the web-server [[http://eschulte.github.io/emacs-web-server/index.html#Top][manual]] and work through other [[http://eschulte.github.io/emacs-web-server/Usage-Examples.html#Usage-Examples][Usage Examples]].
 260
 261 ** Running behind a proxy
 262 Public-facing instance of the Emacs web-server should be run behind a
 263 more established web server such as [[http://httpd.apache.org/][Apache]] or [[http://wiki.nginx.org][Nginx]] to provide
 264 additional robustness and security.
 265
 266 The following example Apache configuration may be used to have a
 267 public facing Apache server listening on port 80 proxy requests to a
 268 local web-server instance running on port 8888 of the same machine.
 269
 270 #+begin_src conf
 271   <VirtualHost *:80>
 272     ServerName  yourserver.com
 273
 274     ProxyPass / http://localhost:8888/
 275   </VirtualHost>
 276 #+end_src
 277
 278 A similar Nginx configuration is available at
 279 http://wiki.nginx.org/LoadBalanceExample.
 280
 281 ** Running behind an https proxy
 282 The following example configurations will cause Apache or Nginx to act
 283 as an HTTPS proxy for an instance of the Emacs web server running on
 284 the same machine.  With this setup Apache speaks HTTPS to the outside
 285 world, and communicates with the Emacs web server using HTTP.  This
 286 allows use of HTTPS even though the Emacs web server does not
 287 implement HTTPS itself.  This setup is recommended for any setup, but
 288 should be considered *required* for sites using BASIC HTTP
 289 Authentication.
 290
 291 *** Apache
 292 This requires that Apache has =mod_proxy= and =mod_ssl= enabled, and
 293 that the certificate and key files required for SSL are present.  This
 294 these requirements satisfied, and assuming the Emacs web server is
 295 listening on port 8888 and is running on the same machine as the
 296 Apache web server an Apache virtual host configuration such as the
 297 following.
 298
 299 #+begin_src conf
 300   <VirtualHost *:443>
 301       ProxyPreserveHost On
 302       ServerName yourserver.com
 303
 304       SSLEngine On
 305       SSLCertificateFile /etc/ssl/certs/ssl-cert-snakeoil.pem
 306       SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
 307
 308       ProxyPass / http://localhost:8888/
 309       ProxyPassReverse / http://localhost:8888/
 310   </VirtualHost>
 311 #+end_src
 312
 313 *** Nginx
 314 See the following for instructions configuring Nginx as an HTTPS
 315 proxy.
 316 - http://wiki.nginx.org/SSL-Offloader#sslproxy.conf
 317 - http://www.cyberciti.biz/faq/howto-linux-unix-setup-nginx-ssl-proxy/
 318
 319 ** COMMENT documentation for running in a chroot jail
 320 See https://wiki.archlinux.org/index.php/nginx#Installation_in_a_chroot.
 321
 322 * Bugs [1/1]
 323 ** DONE Sometimes servers don't stop cleanly
 324 - specifically servers with active client process
 325 - maybe also implement a =ws-stop-all= function