jimmyhenderickx.be

Notes of a Drupal developer.

Drupal Migrate memory issues using drush?

For a project at Intracto digital agency, we are doing daily imports and updates using the fantastic Drupal Migrate framework.

Lately we ran into an issue where our daily migrate updates ran out of memory.   We are running our migrate classes over drush using a cronjob.
Drupal Migrate should handle memory for you, so why was it failing on us?

 

The fix?

We are running php in cli (command line interface) over unlimited memory using a bash script.
Putting a cap on our cli php memory suddenly ensures migrate is managing memory (and respawning a new process when necessary) for us.

Detailed bash-script example for debian-based users:

#!/bin/bash
 
# Ensure PHP over cli has a memory limit when running Drupal Migrate
# else Drupal Migrate wont be able to respawn a new process!
# See: http://www.jimmyhenderickx.be/drupal-migrate-memory-issues-using-drush
 
# Tweakable stuff
php_memory_limit="128M"
drush_path="/usr/local/share/drush/"
drupal_path="/var/www/yourdrupalproject/htdocs/"
 
 
# Run Drupal Migrate imports
php -dmemory_limit=${php_memory_limit} ${drush_path}drush.php mi YourMigrateClass --feedback="1000 items" --root=${drupal_path}

 

Happy migrating!

 

Update: Thanks @tvlooy for suggesting to set memory limit at runtime instead of using php.ini

How to turn off cron in development environments.

Since Drupal 7, poormanscron (auto cron every interval) is in core, which is great. When working on multiple projects in a development environment, I like to turn off this feature though. I dont want every project to load for 5 secs the first time I'm accessing it after a few days working on another project. You could change the cronsettings on admin/config/system/cron I prefer adding the next line to your development settings.php

$conf['cron_safe_threshold'] = '0';

By the way, I do not recommend to do this on live sites

Quicksearch Drupal.org in Chrome

As a Drupal Developer, I'm querying the d.o api and modules quite often. I grew tired of clicking the correct bookmark, and then firing a search, so I tweaked my chrome.

In chrome, go to this url:
chrome://settings/searchEngines

Scroll down, at the bottom you can add your personal "search engines".

I added these 2:

Drupal 7 API - drapi - http://api.drupal.org/api/search/7/%s
and
Drupal Modules - drmod - http://drupal.org/search/apachesolr_multisitesearch/%s?filters=ss_meta_t...

Typing "drmod views" in your addressbar will now search on d.o on all views related modules.

Gogogo!

Easily switching sessions

I'm working on a project which has many different user-roles (Django framework) and find myself logging out and in too frequently.
So I digged up a firefox plugin I havent used for some time: CookieSwap.

Login to a user, switch to another profile, login with another user.
Tadaa, you can now switch between 2 users in 1 second!

 Go Go PowerUser!

Small things do mather: instant filter

Enabling/Disabling a module in drupal can be annoying, if you have lotsa modules, as Ctrl-F will stop at each 'required' hit too.

Instant Filter makes you find your module instantly (Whoa, what an appropriate module name!), by grouping only the modules containing your search string.

Powerusers: file this in: drush make default install

Link: http://drupal.org/project/instantfilter

UPDATE:
Forget about instantfilter
module_filter is what you're looking for

Link: http://drupal.org/project/module_filter

go!

Backup and Migrate just went bananas.

I've just found out Backup and Migrate now supports direct ftp upload and even external database import (if its accessible from external source).  Deploying to shared dev-hostings without drush now is a lot easier.

ne thing I encountered while deploying this way, is that Drupal stores the default files-path in his database. In my opinion its much more clean to set this in your settings.php for your different environments. This is achieved by using this code in settings.php

 

$conf['file_public_path'] = 'sites/default/files';
$conf['file_private_path'] = 'sites/default/files/private';
$conf['file_temporary_path'] = '/tmp';

Am I really doing something on this site?

I'll try to add content on this site if something worthwhile sharing hits me.

Tags: