article on symfony svn externals

for projects i use subversion as the scm for symfony source

http://echodittolabs.org/blog/2009/09/symfony-and-svnexternals-super-slick-easy-way

pressflow a drupal branch for scaling

basicly these guys rebranded drupal with some core hacks that allow it to scale out database reads. i did a diff of the two files a while back and it looked very similar to the following drupal patch.

http://fourkitchens.com/pressflow-makes-drupal-scale/downloads

//database.inc
function db_query($query) {
$args = func_get_args();
array_shift($args);
$query = db_prefix_tables($query);
if (isset($args[0]) and is_array($args[0])) { // 'All arguments in one array' syntax
$args = $args[0];
}
_db_query_callback($args, TRUE);
$query = preg_replace_callback(DB_QUERY_REGEXP, '_db_query_callback', $query);

//load balancing
if(strpos(strtolower($_GET['q']),"admin") !== false)
db_set_active('write'); //its important that all admin gets access to the most recent data
else
if(strpos(strtolower($query),"select") === 0){
db_set_active('read'); //this will not contain any data from the master (write) database untill replication happens
}
else {
db_set_active('write');
}

return _db_query($query);
}

//sites/default/settings.php
//$db_url = 'mysql://username:password@localhost/databasename';

$db_url = array(
'default' => 'mysql://username:password@localhost/databasename',
'read' => 'mysql://username:password@localhost/databasename',
'write' =>'mysql://username:password@localhost/databasename'
);
?>

I got that patch from the following post on drupal.org:

http://groups.drupal.org/node/2147

2bits posts a lot on drupal performance

need to read some more of these. most stuff is known material but im sure there are some gems in there:

http://2bits.com/articles/drupal-performance-tuning-and-optimization-for-large-web-sites.html

article on performace monitoring lamp stack

this article could be extrapolated to things other than lamp. missing are mentions of things like nagios and zabbix both of which are the equivalents of what they are using cacti for in this article. i prefer nagios.

http://2bits.com/articles/tools-for-performance-tuning-and-optimization.html

Drupal Uploading files to folder outside of the webroot

for security reasons you shouldn't allow users to upload files to web accessible directories. I compiled these links after doing some research on how to change drupal's default behavior to allow files to be uploaded to a web inaccesible directory.

Kind of step by step:
http://www.vmtllc.com/drupal-as-an-intranet

Description of drupal public/private filesystem
http://drupal.org/node/230984

Ebook on files in drupal:
http://11heavens.com/files-in-Drupal

List of all file upload modules
http://groups.drupal.org/node/20291

using nginx as reverse proxy with http-acceleration (caching)

I had to use nginx on a project recently rather than varnish (needed ssl, virtual hosts etc) but I also needed to accelerate the drupal app behind the proxy because drupal can be soooo slow. here are some methods that i researched. i think i am going to use the first because it is easiest and most familiar (similar to what varnish does)

Proxy_cache methods (most like varnish):

http://www.ruby-forum.com/topic/183590

Nginx docs for proxy_cache:
http://wiki.nginx.org/NginxHttpProxyModule#proxy_cache

Proxy_store method:

Another older type of nginx caching (well its not really caching more like mirroring flat file copies of content) that just stores a copy of the requested uri's output from the upstream server to disk and then serves that copy indefinitley as long as it exists: (great for sites with content that does not change much)

http://lucasforge.2bopen.org/2009/09/caching-dynamic-content-using-nginx/

More info on using something like the above but this post includes a cron script to delete files from the cache after a certain length of time.

http://mark.ossdl.de/2009/07/nginx-to-create-static-files-from-dynamic-content/

The above 2 posts could be combined with a post like this (on nginx and memcachce) to store the repsonses in memcache rather than on disk:
http://www.igvita.com/2008/02/11/nginx-and-memcached-a-400-boost/

The question would be how to create propper key's into the memcache. I think that you could crc the output from the upstream servers and use that concatenated to the full uri for the object as the key. That way when the content changes the crc will change thus invalidating the old key and the old content that was stored with the old key.

Here are some posts about how to create keys for memcached content that changes and invalidating stale content:

http://blog.leetsoft.com/2007/5/22/the-secret-to-memcached

http://nubyonrails.com/articles/about-this-blog-memcached

Nginx docs for proxy_store:
http://wiki.nginx.org/NginxHttpProxyModule#proxy_store


Alternate caching plugin for nginx:

This project is interesting and would be awesome but looks too experimental and I don't read chinese :(

http://code.google.com/p/ncache/