Pasar URL a minúsculas con nginx

  • english
  • spanish

With nginx and its embedded Perl module it is possible to automatically rewrite/redirect every incoming request to a lowercase URL with something like this:

http {
	...
	perl_modules  perl/lib;

 	perl_set $uri_lowercase 'sub {
   		my $r = shift;
   		my $uri = $r->uri;
   		$uri = lc($uri);
   		return $uri;
 	}';
	...
}

server {
	...

	location / {
               if ( $uri != $uri_lowercase ) {
                       rewrite . http://$host$uri_lowercase;
               }
# or just:
#		rewrite . $uri_lowercase;
	}
	...
}

The first option (with the if and http://$host) sends an HTTP redirect to the lowercase URL if the requested URL is not completely in lowercase. I like this approach better as it kind of normalizes all URL to a canonical form. The second option (without if nor http://) just accepts any URL and internally rewrites them to lowercase: if you rename all your directories and files to lowercase, this will imitate Windows’ case-insensitive behaviour.

Kudos to the following post on the nginx users list were I draw the “inspiration” :D from to get this done. The embedded perl doc and examples are indeed scarce. :-(

http://forum.nginx.org/read.php?2,39425,39944

Anyway I see two problems in this approach:

  • you need the embedded perl module which according to the docs is experimental and can lead to memory leaks.
  • the actual redirection is done with the rewrite, which you can put on the location you need. But the URL lowercase calculation, being on the “http” section of the config, is done for each and every request arriving to your server. Say you have a virtual server with 10 domains and you only need this on one particular location of one of them. The lowercase URL is going to be calculated for every request of every domain.

I guess that by defining a perl function the URL calculation could be restricted to a particular location, have to look into it further. Another option is writing a C module.

And why would anybody want to rewrite every URL to lowercase? We’re migrating a legacy Tomcat-based application from Windows to Linux, and it seems the original developers were not “case-aware” at all, they mixed upper and lowercase in the filenames without regarding how the actual files were named. On Windows this is not a problem but when moving the app to Linux it is. So we have renamed every file and directory to lowercase and used the previous configuration on the nginx servers that load-balance our Tomcat farm. Some corner-cases remain (includes in JSP files, which don’t route back to the nginx server and are dealt internally by Tomcat) and have been dealt with one by one, but the bulk of wrongly addressed images, videos, links to HTML files, etc. now works.

Ajustar la hora en servidores Amazon EC2

  • english
  • spanish

Amazon EC2 virtual servers’ internal clock is synchronized with that of their host server. But some host servers seems to be off-date: I’m working on a four-server cluster (will grow to eight when we go into production) and while three of them were perfectly synchronized, the fourth one lagged behind a couple of minutes.

By default you can’t adjust the date with either date nor ntp, the commands execute but do nothing. In order to decouple the VM’s clock from the host’s one and being able to set the time on your system you need to run the following command:

echo 1 > /proc/sys/xen/independent_wallclock

Bear in mind that, as pointed out here, after decoupling your clock from the host’s you can’t go back, you’ll be on your own forever.

Mantener las sesiones SSH abiertas

  • english
  • spanish

Sometimes SSH sessions die. At work I usually have several ssh sessions with a number of servers, and depending on the configuration of some intermediate router/firewall some TCP sessions get closed after a little while, the SSH sessions die and you have to log back in again.

To prevent this you can instruct sshd to implement some kind of keepalive mechanism preventing the TCP sessions from dying, by including these options on the sshd_config file:

KeepAlive yes
ClientAliveInterval 60

Parche para autenticar nginx con Amazon S3

Vuelvo tras el paréntesis del CCNA (4 módulos aprobados, sólo me falta el examen final de certificación) con algo que he estado haciendo esta semana en el curro: un parche poder usar la autenticación de S3 al hacer de proxy con nginx.

En el curro estamos montando una arquitectura de servidores con EC2, S3 y ELB: los EC2 están distribuidos por capas, con servidores nginx (web y proxy) delante de los servidores de aplicaciones Tomcat, y aparte en S3 tenemos todo el material “pesado” (fotos, vídeos, descargas, etc.) Como nginx puede hacer de proxy, en lugar de mandar a los usuarios directamente a S3 hacemos de proxy y mapeamos el contenido en nuestro propio dominio, con lo que además nos ahorramos unas pelillas (S3 aparte de por ancho de banda y espacio ocupado, cobra por número de conexiones).

El problema viene porque tenemos una gran mayoría de archivos que son públicos (imágenes, iconos y otros elementos del diseño y archivos multimedia) y otros que sólo queremos que los puedan descargar los usuarios registrados en la página. Un paso hacia la solución es usar el parche Secure Download, que genera URL que expiran tras unos minutos con lo que los elementos privados estarían asegurados, pero sólo a través de nuestra página. Si alguien descubre el nombre de nuestro bucket y el esquema de URL (y no es muy complicado….) podría descargar cualquier cosa directamente de S3.

Con este parche se puede acabar de securizar las descargas: se hacen privadas en el bucket de S3, de forma que sólo autenticándose puedan descargarse (el usuario final ya no podría descargarlas aunque averigüe la URL), y nginx si que puede acceder autenticándose a la vez que protege en local las URL con el Secure Download.

El parche está disponible aquí: nginx proxy – Amazon S3 autentication