Creating a sanitized Drupal database dump

greggles — Sat, 27 Mar 2010 15:06:55 +0000

People often want to create a backup copy of their site database and give it to someone else to create an environment similar to the live environment for testing or development. However, doing so exposes all of your site data being leaked if that backup copy is ever placed on a CD that gets lost or a harddrive which is not destroyed at the end of its life or a laptop which is stolen.

I blogged about one solution to this problem earlier with using Virtualbox and an encrypted disk image on Mac for easier and more secure development. However, another simple solution exists: munge the data.

Sanitizing a Drupal database export

The basic idea of this strategy is five easy steps:

Export your database
Import it into a new temporary place
Overwrite sensitive information with dummy data
Export from the temporary place
Share this new export

-- CAUTION: DO NOT RUN THIS ON DATABASE WHERE YOU CARE ABOUT THE INFORMATION!!!

-- Munge emails for security.
UPDATE users SET mail = CONCAT(name, '@localhost'), init = CONCAT(name, '@localhost'), pass = MD5(CONCAT('MILDSECRET', name));
UPDATE comments SET mail = CONCAT(name, '@localhost');
UPDATE directory SET mail = CONCAT(name, '@localhost');
UPDATE authmap SET authname = CONCAT(aid, '@localhost');
UPDATE client SET mail = CONCAT(cid, '@localhost');
-- Only important if you use project and project_issue module
UPDATE project_issue_projects SET mail_digest = 'foo@localhost', mail_copy = 'foo@localhost';
UPDATE projects SET mail = CONCAT("empty", '@localhost');
-- Only important if you use simplenews module
UPDATE simplenews_subscriptions SET mail = CONCAT(snid, '@localhost');

-- Clear out old webform entries which likely include e-mails
UPDATE webform_submitted_data set data='scrubbed';

-- Get rid of irrelevant data it contains IP addresses and bulks up the database
TRUNCATE accesslog;
TRUNCATE access;
TRUNCATE cache;
TRUNCATE cache_filter;
TRUNCATE cache_menu;
TRUNCATE cache_page;
TRUNCATE devel_queries;
TRUNCATE devel_times;
TRUNCATE flood;
TRUNCATE history;
TRUNCATE search_dataset;
TRUNCATE search_index;
TRUNCATE search_total;
TRUNCATE sessions;
TRUNCATE watchdog;

-- Alter sensitive entries in the Variable table
update variable set value = 's:4:"fake";' where name = 'smtp_password';

The above script is a slightly modified version of a fairly specific script that works for one specific database: drupal.org. However, you can see the techniques used here and imagine how to adapt them for other sites.

Modify the script to protect data for your specific site

One important step that I take when adapting this to a new site is to make my export from step 4 above and then do a search through the text of the export for common e-mail providers like yahoo or gmail to see if any more addresses exist. If you find one then you figure out which column they are in and use a similar "munge" process on the column updating it to null.

Take special care with some modules like twitter that may store account credentials (twitter usernames and passwords) in a database table.

And take extra special care with modules like smtp or mailhandler which store credentials for important email accounts in the variable table or other locations in the database.

Once you've got this script tuned for your site you may need to update it as you add new modules or upgrade existing modules.

And that's it - now you can share copies of your database with greater confidence that your site won't be the source of a data leak.

Edit: Updated to include webform example based on jbrauer's similar blog post.

Cracking Drupal - leaking data

Creating a sanitized Drupal database dump

Sanitizing a Drupal database export

Modify the script to protect data for your specific site