Archive

tail -f access.log … ¡gráfico!

A la izquierda los clientes, a la derecha las páginas que va entregando el servidor, la “raqueta del pong” representa al servidor atendiendo a cada cliente, y el nº que rebota junto con cada bola (petición) el código HTTP que devuelve. ¡Impresionante! ¡Yo quiero! :-D


YouTube

¡Gracias Meiziu! ;-)

Enviar a:
  • Digg
  • Sphinn
  • del.icio.us
  • Facebook
  • Mixx
  • Google
  • BarraPunto
  • Meneame
  • Slashdot
  • Technorati
  • YahooMyWeb

Verification fields validation in CakePHP

  • english
  • spanish

When designing a registration form for a web site, it’s usually a good idea to include verification fields for both the password and the e-mail address: a typo on the password would leave the user unable to log into the system, while one on the e-mail address would prevent us from reaching the user. Asking twice for these fields allows us and the user to verify that there are no typos and that both values are correct, thus avoiding future problems.

With CakePHP these verification fields can’t be validated automatically with the model’s definition rules, it has to be done programmatically on the controller as there is no model verification rule to check that an object’s field’s value equals that of another field (equalTo compares against a string as a literal, not against a variable).

By defining the following function on the model we are defining a custom verification rule that will allow us to do this verification properly on the model (or download it here, the plugin I use for code highlighting or wpmu itself keep trashing all > symbols):

  1. <pre>        /**
  2.          * Verifies that the field beeing validated matches the value of the
  3.          * field in the second parameter. If omitted and the field beeing
  4.          * validated’s name is NAME_verification, the value will be compared
  5.          * to that of the NAME field.
  6.          *
  7.          * Examples:
  8.          * <code>
  9.          * var $validate(
  10.          *      ‘password_check’ =&gt; array(
  11.          *              ‘rule’ =&gt; array(’verifies’, ‘password’),
  12.          *              ‘message’ =&gt; ‘The passwords don\’t match’
  13.          *      ),
  14.          *      ‘email_verification’ =&gt; array(
  15.          *              ‘rule’ =&gt; array(’verifies’),
  16.          *              ‘message’ =&gt; ‘The email addresses don\’t match’
  17.          *      ),
  18.          * )
  19.          * </code>
  20.          *
  21.          * The second case will verify automagically against the "email" field.
  22.          */
  23.         function verifies($data, $field=null) {
  24.                 $keys = array_keys($data);
  25.                 $key = $keys[0];
  26.                 if(!is_string($field)) {
  27.                         if( ($pos = strpos($key, “_verification”)) === FALSE ) {
  28.                                 return FALSE;
  29.                         }
  30.                         $field = substr($key, 0, $pos);
  31.                 }
  32.                 return ($data[$key] == $this-&gt;data[$this-&gt;name][$field]);
  33.         }</pre>
Enviar a:
  • Digg
  • Sphinn
  • del.icio.us
  • Facebook
  • Mixx
  • Google
  • BarraPunto
  • Meneame
  • Slashdot
  • Technorati
  • YahooMyWeb

Astronomía básica para torpes

Vía microsiervos un pequeño curso de introducción rápida a la astronomía, para identificar rápidamente un par de constelaciones y un par de estrellas, y saber orientarse buscando el norte con la Estrella Polar.

La astronomía es algo que siempre me ha llamado bastante la atención pero nunca le he prestado un mínimo de interés como para aprender algo más allá de identificar la Osa Mayor y poco más, y eso que mi amigo Gabi es un auténtico frikazo del tema y me he ido con él y el telescopio un par de veces.

Enviar a:
  • Digg
  • Sphinn
  • del.icio.us
  • Facebook
  • Mixx
  • Google
  • BarraPunto
  • Meneame
  • Slashdot
  • Technorati
  • YahooMyWeb

Reloj arreglado

Pues parece que he tenido algo de suerte con el tema del reloj: ayer me llamaron de la relojería, que ya estaba arreglado (again) y se hacían cargo de los gastos.

Y digo que he tenido suerte porque el lunes llamé a la oficina de atención al consumidor de Las Rozas y me dijeron que la cosa estaba chunga: lo único cierto con lo que estábamos de acuerdo ambas partes era que me llevé el reloj bien y lo devolví al día siguiente roto, y en principio no había forma de probar si le había dado un golpe o lo que fuera y era culpa mía la rotura, que íbamos a necesitar un informe del taller que lo arregló (¿y qué hubieran dicho? ¿”si, es culpa nuestra, somos unos palotes”?) En pocas palabras, que si se hubieran puesto chungos lo más seguro es que me hubiera tocado aguantarme. :-/ Qué poco protegidos estamos a la hora de la verdad como consumidores, ¿no?

Enviar a:
  • Digg
  • Sphinn
  • del.icio.us
  • Facebook
  • Mixx
  • Google
  • BarraPunto
  • Meneame
  • Slashdot
  • Technorati
  • YahooMyWeb

SCAM inmobiliario

No, no voy a hablar de las hipotecas subprime (buenísimo el vídeo, Vichu), si no de otra estafa más en Internet.

Siguiendo con lo de aprovechar la baja para algo útil llevo unos días mirando en portales inmobiliarios pisos en el centro de Madrid, que ya llevo bastante tiempo encerrado en mi cueva en Las Rozas. A ver si me voy de una vez al centro, al meollo de la ciudad, y me aireo un poco que falta me hace.

El caso es que hay alquileres que van de lo carísimo, pasando por lo caro, lo casi razonable… hasta lo ridículamente barato para las características del piso siendo Madrid. Y aparte de pedir información y llamar a alguno de los razonables (mañana voy a ver un par) he pedido información de dos de los sospechosos, a ver qué pasaba. Os pongo enlaces a los anuncios y el mail de respuesta:

Good day,
I have available the apartment located in Calle de Isabel 19, 28013 Madrid, as I recently secured a long-term position at FAO headquarters in London and I need money to pay the rent here. This is the reason I want to rent it and also to find a good person for my apartment.
The rent for one month is 550 euro including all utilities(water, electricity, Internet, cable, parking, air conditioning, fireplace, dishwasher, garbage disposal, microwave, refrigerator).
You can move in the apartment in the same day when you receive the keys. The only problem is that I’m in London and I’m the only person who have the keys but I hope that we will find a compromise.

I will look forward to hear from you soon!
Thanks,

Gloria

Hello,
First of all I’m sorry for the late reply but I had a lot to work and this is the time I could give you a message. From the TUCASA add, you could see, that the apartment located in: Calle Mayor 58 ,28013, Madrid,SPAIN. It is located on the 3rd floor, the number of the apartment is #11C. It has 1 bedroom, 1 bathroom, 1 kitchen and 1 living room .
I am looking for someone to live in it, as soon as possible, because my company has won the auction for a project, in construction, for a building that will rise up in United Kingdom, London. I am a civil engineer, so my accommodation period in United Kingdom will be about 4 years. The renting period is from 2 months upto 4 years. As a result of this I want to rent, for this period, at this price and also to find the perfect person for my apartment. I`m the owner of the apt and it’s like in the pics.
The apartment is furnished, but I have the option of sending all my furniture into storage if you want to bring your own (no extra costs). The rent for 1 month is 550 Euro (for the whole apartment) including all utilities (water, electricity, Internet, cable, parking, air conditioning, fireplace, dishwasher, garbage disposal, microwave, refrigerator). You can move in the apt in the same day when you receive the keys. The only problem is that I`m the only person who has the keys but I hope that we will find a solution.

Thank you for your interest and I will look forward to hearing from you soon.

Good luck,
Jose Banderas
P.S. Please tell me if this is what are you looking for and reply my only if you are truly interested in the aptartment.

¿Cómo os han dejado el cuerpo? Al más puro estilo del SCAM nigeriano, ¿no?

¿Por qué José Banderas (¿hermano de Antonio?) me escribe en inglés? Ah, que tengo el traductor automático, era Joseph Flags…

¿No mosquea el copy-pasteo brutal del “including all utilities (water, electricity, Internet, cable, parking, air conditioning, fireplace, dishwasher, garbage disposal, microwave, refrigerator)”? Con la ilusión que me hace tener un fireplace. :-(

¿No es sospechoso que los dos correos acaben con una frase prácticamente idéntica letra a letra, diciendo que seguro que encontramos una solución a que él tenga las llaves? ¿Y la misma excusa de que le ha salido un proyecto en Londón (de las Nieves)?

Nfin…

Enviar a:
  • Digg
  • Sphinn
  • del.icio.us
  • Facebook
  • Mixx
  • Google
  • BarraPunto
  • Meneame
  • Slashdot
  • Technorati
  • YahooMyWeb

Mensaje al receptor equivocado

¿No estáis hasta los mismísimos de los anuncios contra la piratería en los DVD? Me parece algo bastante absurdo: si he comprado el DVD, aparte de que por eso mismo ya no soy el destinatario del mensaje, ¿por qué me tengo que tragar el minuto y pico de rollete sin poder saltar al menú? Y si me he bajado la peli del torrent, será directamente la peli en DivX (sin menú ni extras ni el dichoso anuncio) o bien en DVD que alguien se habrá encargado ya de “limpiar” de contenido no deseado… Es decir, el mensaje al final sólo llega ¡¡al que no ha pirateado la película!!

Si es algún tipo de refuerzo positivo la verdad es que preferiría que bajaran los precios, que cuando preparan una nueva edición mega-extendida-director’s-cut de alguna peli no se cortan en bajar el precio de la anterior (extendida a secas, pero igualmente cargada de extras en 2 DVD) a un tercio o menos de lo que costaba unos meses antes. O cuando después de sacar una serie en varias entregas de un DVD cada una, luego sacan el pack de la serie completa con 6-8 DVD por lo que costaban dos de los DVD individuales. Eso NO es refuerzo positivo, desde luego. Lo que refuerza son las ganas de mandarlos a fer la mà y la próxima vez no comprar el DVD.

Por no hablar de los logos del Ministerio de Cultura y el de Justicia al final… ¿eso significa que todo ésto se está costeando con mis impuestos? Porque preferiría que los invirtieran en cosas más útiles. O ya puestos que me desgravaran por cada DVD original que compre, que para eso estoy colaborando con la cultura, ¿no?


YouTube

Enviar a:
  • Digg
  • Sphinn
  • del.icio.us
  • Facebook
  • Mixx
  • Google
  • BarraPunto
  • Meneame
  • Slashdot
  • Technorati
  • YahooMyWeb

Radiografías antes y durante

Quería hacer el típico “antes … después” pero aún no me han hecho ninguna radiografía sin el fijador. Así que supongamos que esta es la primera parte de una serie de dos. ;-)

Antes (radiografía y detalle de la resonancia, el mismo día del accidente):

Durante (última que me hicieron con el fijador):

Enviar a:
  • Digg
  • Sphinn
  • del.icio.us
  • Facebook
  • Mixx
  • Google
  • BarraPunto
  • Meneame
  • Slashdot
  • Technorati
  • YahooMyWeb

El cristal que se rompe solo

Estoy mosqueado. Un mosqueo tamaño familiar.

<rant>

Tengo un reloj Lotus Titanio hace unos 8-10 años que los que me conocéis estaréis hartos de ver porque lo llevo a diario. En estos años se ha llevado golpes, caídas, arañazos, presión (30m buceando = 4 atmósferas)… sólo tenía un par de picados en el borde del cristal, superficiales.

Con el accidente se llevó parte del golpe, la muñeca que me rompí es la izquierda, donde llevo el reloj, y acabó con varios arañazos en el cristal. No muy profundos, ninguno atravesaba el cristal por completo, pero bueno… fue culpa del accidente, así que igual que el seguro me va a arreglar los arañazos de la moto, que me arregle los del reloj, ¿no?

La semana pasada lo llevé a una relojería, y me llamaron ayer, que ya estaba. Lo recogí por la mañana, me lo puse, y por la tarde me doy cuenta que tiene una raja de lado a lado del cristal, una línea recta perfecta, sin aristas desportilladas ni nada (ni ningún otro arañazo), y profunda, todo el espesor del cristal. Ni que decir tiene que no le he dado ningún golpe (en la muñeca izquierda me hubiera dado cuenta yo mucho antes que el reloj, os lo aseguro), y aún así ¿qué tipo de golpe le tendría que haber dado para hacerle ESO? Así que esta mañana he vuelto a la relojería.

No estoy mosqueado porque se haya roto. Shit happens. Estoy mosqueado por el trato del dependiente. Desde el primer momento cuando le he dicho “mira lo que ha pasado” su expresión y sus palabras eran de “me la suda!” y de dudar de mi palabra: “y qué queires que haga yo”, “ayer estaba bien”, “quien no le he dado un golpe he sido yo”… WTF?!

Al final se lo ha quedado pero con un “lo mando al taller a ver qué dicen, pero quien no lo va a pagar soy yo”. ¿Cómo que no? Ahora estoy dudando de si he hecho bien en dejárselo a ver si los del taller tienen algo más de tacto y reconocen … que el cristal tenía un defecto, que se han pasado apretando o lo que sea, o si debería haber ido directamente a consumo o donde toque a ponerles una denuncia (algo que voy a hacer como no me lo arreglen, está clarísimo).

Por cierto, otra joya de éste hombre: aparte de cambiar el cristal le hicieron unas pruebas especiales de presión y tal para ver que aguantara al bucear con él. Pues me ha soltado algo en plan “pues el cristal no sé, igual te lo cambian, pero ¿ahora hay que hacer otra vez las pruebas de presurización? ¿Y quién las paga?” En serio, yo ALUCINO de la profesionalidad y el saber estar cara al cliente de algunas personas.

</rant>

Enviar a:
  • Digg
  • Sphinn
  • del.icio.us
  • Facebook
  • Mixx
  • Google
  • BarraPunto
  • Meneame
  • Slashdot
  • Technorati
  • YahooMyWeb

Jedi EcoSaber

Visto en Microsiervos.

Enviar a:
  • Digg
  • Sphinn
  • del.icio.us
  • Facebook
  • Mixx
  • Google
  • BarraPunto
  • Meneame
  • Slashdot
  • Technorati
  • YahooMyWeb

Open source, scalable MTA clusters

  • english
  • spanish

At my previous job I was responsible for the MTA of a group of companies, handling around 3000 e-mail accounts spread over 20 domains. This MTA received around 150,000 mails daily, and over 95% of them was discarded/marked because it was identified as SPAM or viruses (as of last year, don’t know how this evolved since I left). We used a homegrown cluster of seven servers, which enabled us to scale as needed. And it was based on free software.

This is not an step-by-step installation guide with technical details and configuration files, but rather the story of the evolution of the service, the various problems that we faced, how we solved them, and the design decisions in each case.

Migration

The first incarnation of the server was in 2001 when we had to migrate the old server, which was starting to give lots of trouble, to more current software and hardware. I seem to remember it was a mail server from Netscape (!?) that stored the account information in an LDAP directory, but can’t recall the exact name or version of the product. The server we chosed for the migration was qmail-ldap, mainly because of the good reviews we read about its stability, reliability and security, ease of setup (personally I still think qmail is much simpler than eg sendmail) and because it also used an LDAP directory. The latter may seem a silly reason, but in the end the migration had to be done in extremis at a time that the original server wouldn’t even boot most of the times, and we got away with it with a simple ldapsearch and a little script that “translated” the LDAP scheme of one server into that of the other one. Over time the choice of qmail-ldap proved to be the right one, because thanks to its modular design it allowed us to progressively move from a one server deploy to the cluster that I refered about in the introduction.

This first server was a rack-mounted one, with redundant power supplies and hw RAID5, so that all the data was secure (or so we thought back then). We also rolled qmail-scanner and the Kaspersky anti-virus (there was no ClamAV yet, we moved to it some years later). The same server held the SMTP, POP, IMAP and WebMail (SquirrelMail) services.

Active/Passive backup

We had to do the first architectural upgrade a couple of months after the migration: a RAID5 hiccup lead to a corrupted filesystem which was quite difficult to fix. It became clear that the RAID discs and the redundant power supplies were not enough to ensure the data integrity and service availability, so we installed another server exactly like the first one, and synchronized the configuration and mailboxes using rsync and cron jobs. The switching from the primary to the backup server was manual back then, using NAT at the router.

Over time the server was upgraded to new models several times, but we kept the active/passive backup structure. The syncronization between both servers was also improved, with DRBD for the mailboxes and csync2 for the configuration, AV bases, and so on. Master-backup monitoring and service switch was automatized with heartbeat.

The SPAM flood, specialization by resources

Sometime around 2002-2003 viruses ceased beeing e-mail’s biggest problem: the increasing number of SPAM messages received every day was way worse. So we threw SpamAssassin into the mix. Over time this lead to an ever-increasing CPU and memory consumption, slowing the server to a crawl. At first it seemed that the only option was to migrate every year to a new, more powerful server (and what would we do with the old one then?), or have multiple servers and distribute all the domains among them in an attempt to distribute the load.

Finally we realized that we had two different kinds of resource needs, with different growth patterns:

  • HD space for the mailboxes: the number of mailboxes in our system was fairly stable and the vast majority of our users downloaded their e-mails using POP, so HD scalability wasn’t really that big of a problem for us. We could easily afford to upgrade disk every few years, moving the service to the backup server while we were upgrading the master one.
  • CPU for the filtering: SPAM was growing at an exponential rate, we basically needed to double the CPU power each year.

So, why not specialize our servers into storage servers and a filtering farm? We moved the SMTP service from the main servers to a front-line of SMTP servers with the follwing characteristics:

  • they were off-the-shelf PCs and their configuration was practically identical (no variations appart from hostnames and IP addressess). We prepared a system image we could easily dump in a matter of minutes to a new PC, in case one of the servers went down or we needed more raw CPU power because of an increase in SPAM.
  • we had a router load-balaincing port 25 among all these servers.
  • all these SMTP servers were independent from the central ones, except for the final step of delivering the already analized mail to its destination mailbox: each server had a local copy of the LDAP directory (synchronized with slurpd), a copy of all the configuration files and all the AV bases and the SpamAssassin bayesian database (synchronized with csync2), and a DNS resolver/cache (dnscache).
  • they did local logs, but also sent them to a centralized syslog server for easier analysis.
  • they didn’t store the mails locally for later delivery, in other words they had no delivery queue: e-mails were analyzed on the fly during the SMTP session and if one of them met certain anti-SPAM/AV criteria (blacklisted IP, a number of RBL hits, certain keywords, etc.) it was immediatelly rejected with an SMTP error and the connection was closed; on the other hand if the mail was let through (it was either legitimate, or marked as possible SPAM), it was sent to the central server on the spot, and the filtering server never gave the OK to the origin MTA until the mailboxes server acknowledged the delivery. This is done quite simply with qmail by means of replacing the qmail-queue binary with the qmail-qmqpc one. By doing this we were able to guarantee that no mail would be lost in the event that a filtering server crashed, as the origin MTA wouldn’t receive the OK from us and would re-try the delivery after a couple of minutes.

Mailboxes, the POP and IMAP services, the LDAP master, webmail, and the remote queue remained in the central server, although most of them could have been moved to independent servers if needed, but we never needed to.

Specialization by type of client

The next problem we faced came about 2-3 years ago when image- and PDF-based SPAM became popular: we added an SpamAssassin plugin which re-composed animated GIF images and did OCR to all image attachments. This extra analysis greatly increased our CPU needs (we had to go from 2 or 3 filtering servers to 5 in a couple of days) and even so there were times when a server got overloaded for some 5-10 minutes and an e-mail could take not less than 2 minutes to be processed, delivered and SMTP-OK’d. When this happened and the sending party was another MTA it represented no bigger issue, as in the event of a timeout or disconnection the remote server would re-try the delivery several times; however, if the sender was an end-user with his MUA, a longer-than-usual delivery time or (God forbid) an error message from Outlook because of an eventual dropped connection lead to a phone call to the IT team because “the mail wouldn’t work.” :-)

The solution was splitting the SMTP and analysis farm into two: one for external mail and another for internal ones, for our users. The first farm is the one the DNS’ MX records pointed to, and had all the SPAM filtering options activated; while the second one retained the domain name end users used as the SMTP server in their MUAs, had all the heavy-weight lifting filters disabled and required SMTP authentication (wouldn’t accept non-authenticated sesions even for local domains). This way all external e-mail coming from remote MTAs would go through all the filters, and our users went to the privileged servers with somewhat lesser filering capabilities (but enough for internal mail) and great response times.

The big picture

Enviar a:
  • Digg
  • Sphinn
  • del.icio.us
  • Facebook
  • Mixx
  • Google
  • BarraPunto
  • Meneame
  • Slashdot
  • Technorati
  • YahooMyWeb