At Torchbox we’ve been working a lot with the Migrate module recently. It’s a framework for representing relationships in the data you want to import; both relationships between bits of data, and also relationships between the data and relevant Drupal entities. It used to be a GUI-driven system, but now it seems quite code-heavy.
Which suits us fine, certainly for one-off imports which can be built most straightforwardly by developers. We had to write an import for around 4600 users, plus data which we wanted to store in three Profile2 profiles (address, subscriptions, personal information.)
Migrate did a lot of the heavy lifting for us, along with some code examples. The most useful one was this example import module, started by wusel and edited by (among others) Profile2’s maintainer joachim. As a thankyou I’ve contributed to that post a bit, tidying things up and incorporating some of the lessons we learned.
The results? A flawless migration, including mapping many CSV columns to three multi-valued entity fields for the subscriptions (basically an ORM mapping.) And it was blisteringly fast: user importing on a reasonably high-spec server was at the rate of 12592/min, and we had a peak profile import of 9954/min.
