Accessing a MySQL database from Node.JS

I just pushed node.dbslayer.js to GitHub, a very basic and easy-to-use interface to DBSlayer.

What is DBSlayer?

The DBacesslayer aka DBSlayer is a lightweight database abstraction layer suitable for high-load websites where you need the scalable advantages of connection pooling. Written in C for speed, DBSlayer talks to clients via JSON over HTTP, meaning it’s simple to monitor and can swiftly interoperate with any web framework you choose.

Developed by the New York Times to distribute the load of their MySQL servers, DBSlayer allows us to perform queries in a non-blocking way. Running a SQL query is as simple as:

connection.query("SELECT * FROM TABLE").addCallback(function(result){
	for (var i = 0, l = result.ROWS.length; i < l; i++){
		var row = result.ROWS[i];
		// do something with the data
	}
});

Have fun!

This post has 13 responses

Posted in Blog, Server side 9 months ago

Sending email with Symfony 1.2 and Swift 4

If you had used Symfony 1.2 with Swift 3 before, you probably sent emails like this:

try
{
  $mailer = new Swift(new Swift_Connection_NativeMail());
  $message = new Swift_Message('The subject', 'The body <b>html</b>', 'text/html');
  $mailer->send($message, 'to@user.com', 'noreply@company.com');
  $mailer->disconnect();
}
catch (Exception $e)
{
  $mailer->disconnect();
}

However, due to API changes in Swift 4, that would have thrown an error:

Fatal error: Cannot instantiate abstract class Swift in /[...]/apps/frontend/modules/[...]/actions/actions.class.php on line 40

It turns out that now the class Swift is defined as abstract, which means it can’t be instantiated. Now, classes that you used to instantiate directly, like Swift_Message, have been added factory methods called newInstance

require_once('lib/vendor/swift/swift_init.php'); # needed due to symfony autoloader
$mailer = Swift_Mailer::newInstance(Swift_MailTransport::newInstance());
$message = Swift_Message::newInstance('The subject')
         ->setFrom(array('noreply@company.com' => 'Mailer Name'))
         ->setTo(array('email@email.com' => 'Name Lastname'))
         ->setBody('The body <b>html</b>', 'text/html');  
$mailer->send($message);

Remember that it’s good practise to get the body HTML from a partial, instead of hardcoding it into the action.

$mailBody = $this->getPartial('emailPartial', array(/* parameters */));
// ... 
->setBody($mailBody, 'text/html');

The code has become a lot more readable, easier to customize and write.
More information here and here.

This post has 9 responses

Posted in Server side about 1 year ago

PHP URL Shortening Class released

I’ve just released PHPShortener, a PHP class that makes it very easy to encode/decode URLs with services such as tr.im, is.gd, and others.

Encoding and decoding is a simple as this:

$s = new PHPShortener();
// encode a long url
$shorturl = $s->encode('http://devthought.com/projects/php/phpshortener/', 'is.gd');
// decode a short url (autodetects the service)
$longurl = $s->decode('http://tr.im/jBBp');

Head to the project page for downloads and documentation. Fork me on GitHub if you want to contribute!

This post has 11 responses

Posted in Server side about 1 year ago

The Digg worm that wasn’t

Two nights ago I was editing my Digg profile and couldn’t help but think about the recent Mikeyy and Twitter revolt. Within minutes I had found a XSS exploit that could theoretically allow me to achieve the same.

Half an hour later, I had a working worm ready to infect everyone that saw my profile page, which also would propagate to theirs.

Continue reading

This post has 20 responses

Posted in Server side about 1 year ago

Symfony scaling: moving your uploads to a NFS share

A few weeks back a client relaunched a Facebook App that commended me to rewrite. The growth since then has been tremendous, with over 2000 registered users everyday, who upload pictures, make friends and comment on other users’ profiles. My application has been able to handle that growth perfectly in terms of responsiveness, but I was told that if the growing rate persisted, within only a few days the content would exceed the hard drive capacity.

The natural solution for these scenarios is a NFS mount, so that the content can be distributed across a network, which gives your application virtually limitless storage capacity. Joyent provides this service reliably.

The biggest challenge was to minimize the downtime, so these were the steps we took:

  1. Move all current files to the NFS mount to clear up disk space

  2. Override the sf_upload_dir configuration directive. In settings.yml

    all:
       upload_dir:      /nfsmount/site/uploads

  3. Adjust the Apache VHost configuration with an alias that points outside the DocumentRoot (hence the need for a special <Directory>)

    Alias /uploads /nfsmount/site/uploads
<Directory "/nfsmount/site/uploads">
       Order allow,deny
       Allow from all
</Directory>
  4. Clear symfony cache and reload Apache. If your application is well-written, the change should be transparent to all the modules that deal with file uploads.

  5. Use cp -uvR to make sure no images were left out from the initial copy.

This post has 3 responses

Posted in Server side about 1 year ago

Prevent caching of modified Javascript & CSS assets

There’s a very useful PHP function called filemtime, that returns the timestamp of the modification time of the file. This is similar to how the HTTP 1.1 ETag header is generated. The strategy is basically to append the modification date to the script or CSS URI in order to refresh the user’s cache when you’ve modified them.

This is an extract from Devthought header.php Wordpress template file:

<link rel="stylesheet" href="<?php echo get_stylesheet_directory_uri() . '/style.css?' . filemtime(get_stylesheet_directory() . '/style.css'); ?>" type="text/css" media="screen" title="Stylesheet" charset="utf-8" />
 
<script type="text/javascript" charset="utf-8" src="<?php echo get_template_directory_uri() . '/js/scripts.js?' . filemtime(get_template_directory() . '/js/scripts.js'); ?>" ></script>

All you have to do is change the routes to match your files. If you’re not using wordpress, you’ll have to remove the get_stylesheet_directory* and get_template_directory* function calls and replace with your paths.

This post has 3 responses

Posted in Server side about 1 year ago

Custom fields in edit/new admin generator views in Symfony

Let’s say we want to create a field to set the password in plain text from the admin panel, then convert it to sha1 before saving it. Our hypothetical module will be located in backend/modules/admins/, which is a Symfony admin generator module for an AdminUser model.

First we define the getters and setters for the plaintext password in the AdminUser model:

  public function setPasswordPlain($plain)
  {
    if(! empty($plain))
        $this->setPassword(sha1($plain));
  }
 
  public function getPasswordPlain()
  {
    return "";
  }

Then, in generator.yml we add this field to form sections, using the display parameter:

      form:    
        display:              [login, password_plain, email]

If we try the generator now, we’ll get an error message telling us that password_plain widget does not exist, which is true. The new admin generator uses the great Symfony 1.2 forms system, which treats a form as a set of widgets and validators.

If we inspect the lib directory of our generated module we’ll notice the file called adminsGeneratorConfiguration.class.php. This file extends BaseAdminsGeneratorConfiguration which, if we look closely, configures what form is related to this generated module:

  public function getFormClass()
  {
    return 'AdminUserForm';
  }

What we have to do is obvious. We extend AdminUserForm, and we override that method to return the name of our new class. In the same lib directory we create the form:

class myAdminUserForm extends BaseAdminUserForm
{
 
  public function configure()
  {
    parent::configure();
 
    $this->setWidget('password_plain', new sfWidgetFormInput());
    $this->setValidator('password_plain', new sfValidatorString(array('max_length' => 255, 'required' => false)));
  }
 
}

and finally we alter the adminsGeneratorConfiguration accordingly:

class adminsGeneratorConfiguration extends BaseAdminsGeneratorConfiguration
{
 
  public function getFormClass()
  {
    return 'myAdminUserForm';
  }
 
}

This post has 8 responses

Posted in Blog, Server side about 1 year ago

Set up a powerful FTP Server in OS X Leopard

Pure-FTPd has always been my ftp server daemon of choice for UNIX systems. It’s free, secure, fast, widely deployed and definitely more powerful than the built-in /usr/libexec/ftpd.

I’d never tried it on Mac, but it was surprisingly easy to set up.

First, download the latest stable release.

mkdir ~/src
cd ~/src
ftp http://download.pureftpd.org/pub/pure-ftpd/releases/pure-ftpd-1.0.21.tar.gz
tar -xzvf pure-ftpd-1.021.tar.gz
cd pure-ftpd-1.0.21

Although it’s outdated, read the README file for OS X

more README.MacOS-X

Proceed to install. We enable PAM to use the system users for login, and bonjour. If you want other login options, run

./configure --help

You might want to use puredb (for virtual users) instead of your system users or MySQL.

./configure --with-pam --with-bonjour
make install-strip

Unlike GNU/Linux systems, OS X uses the flexible launchd daemon to manage cron jobs, boot tasks and daemons. Since we want to replace the bundled ftpd daemon, we’ll edit the existing plist. Before editing anything, make sure FTP is disabled in System preferences > Sharing > File Sharing > Options.

nano /System/Library/LaunchDaemons/ftp.plist

With the following content

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" 
	"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>Label</key>
	<string>org.pureftpd.ftpd</string>
	<key>ProgramArguments</key>
	<array>
		<string>/usr/local/sbin/pure-ftpd</string>
		<string>--chrooteveryone</string>
	</array>
	<key>Sockets</key>
	<dict>
		<key>Listeners</key>
		<dict>
			<key>Bonjour</key>
			<true/>
			<key>SockFamily</key>
			<string>IPV4</string>
			<key>SockServiceName</key>
			<string>ftp</string>
		</dict>
	</dict>
	<key>inetdCompatibility</key>
	<dict>
		<key>Wait</key>
		<false/>
	</dict>
</dict>
</plist>

This file triggers the Pure-FTPd daemon on demand. Enable FTP in File Sharing, and let’s try to connect:

ftp localhost
Trying ::1...
ftp: connect to address ::1: Connection refused
Trying fe80::1%lo0...
ftp: connect to address fe80::1%lo0: Unknown error: 0
Trying 127.0.0.1...
Connected to localhost.
220---------- Welcome to Pure-FTPd ----------
220-Local time is now 13:02. Server port: 21.
220-IPv6 connections are also welcome on this server.
220 You will be disconnected after 15 minutes of inactivity.
Name (localhost:willy): willy
331 User willy OK. Password required
Password:
230-User willy has group access to:  admin    _appserv _appserv _lpadmin
230- com.appl com.appl staff
230 OK. Current directory is /Users/willy
Remote system type is UNIX.
Using binary mode to transfer files.
ftp>

If you want to customize the many options Pure-FTPd has, edit the ProgramArguments section in the plist file. You’ll definitely want to change it if you set it up with MySQL or puredb instead of PAM. You might want to add chrooteveryone to lock the users to their homes by appending this option to that section, for extra security:

<string>--chrooteveryone</string>

This post has 1 response

Posted in Server side about 1 year ago