With the daily work of the office it is natural that there are challenges and potential research topics. With the addition of our developer Lucas as a Drupal core Migrate system maintainer, we decided to delve into the little documented area of migrating data into paragraphs.
Paragraphs is the new way of creating content. It allows site builders to make things cleaner so they can give more editing power to their end users.
In this blog I will explain in a simple way how to migrate data from CSV into paragraphs using the scenario of a Landing Page content type that has a banner field (single value paragraph) and a tiles field (multi-valued paragraphs field). There is a difference in how you have to structure the migration into single or multi-valued paragraphs fields, so this covers both nicely.
Let's first review our two CSV files. The first line in the files is the header row. Followed by rows of the values of the CSV. Very typical stuff so far.
# files.csv
banner
banner.jpg
img1.jpg
img2.jpg
img3.jpg
# landing_pages.csv
id,title,banner_image,banner_link,banner_link_text,desc1,img1,desc2,img2,desc3,img3
1,Services,banner.jpg,http://www.mtech-llc.com,"Go to site",Desc1,img1.jpg,Desc2,img2.jpg,Desc3,img3.jpg
Next we should build our paragraph types. The banner type has two fields. This should seem very familiar if you have ever dealt with Paragraphs or building a Drupal site.
Paragraphs Type: Banner
Field Name | Field Type |
Call to Action | Link |
Image | Image |
Paragraphs Type: Tile
This tile paragraph type has two fields too and allows someone to print some text with an associated image field. The author of the landing page is going to build a well structured set of tiles for the main content region of the landing page. Of course, you could swap out these fields and make your paragraphs however you want. But this just gives you some very typical paragraph types and migration approaches.
Field Name | Field Type |
Description | Text (formatted, long) |
Image | Image |
Finally, we build our landing page node content type.
Content Type: Landing Page
Field Name | Field Type |
Banner | Entity reference revisions (paragraph field) |
Tile | Entity reference revisions (multi-value paragraph field) |
At this point, we are ready to migrate data. You've already downloaded and installed Entity Reference Revisions and Paragraphs, but you'll also need to install Migrate Plus, Migrate Source CSV and Migrate Tools. And for migrating into Paragraphs, you'll also need to install this patch.
Now we'll build a custom module using Drupal Console. I've named mine paragraph_migration and added dependencies on migrate_plus, migrate_source_csv & migrate_tools. For convenience of this example, inside of this module at paragraph_migration/assets/csv I've placed my CSV files and atparagraph_migration/assets/img I've put my source images. Lastly, I store my migration yaml at paragraph_migration/config/install.
First up, a simple migration of files.csv: For more info on files migrations, head to this earlier blog post: How to Migrate Images into Drupal 8 Using CSV Source
When we are done with the migration, the files will show up in /admin/content/files.
dependencies:
enforced:
module:
- paragraph_migration
id: files
source:
plugin: csv
path: modules/custom/paragraph_migration/assets/csv/files.csv
header_row_count: 1
keys:
- name
constants:
source_base_path: modules/custom/paragraph_migration/assets/img
destination_base_path: 'public:/'
process:
filename: name
source_full_path:
-
plugin: concat
delimiter: /
source:
- constants/source_base_path
- name
-
plugin: urlencode
destination_full_path:
-
plugin: concat
delimiter: /
source:
- constants/destination_base_path
- name
-
plugin: urlencode
uri:
plugin: file_copy
source:
- '@source_full_path'
- '@destination_full_path'
destination:
plugin: 'entity:file'
migration_dependencies:
required: { }
optional: { }
- Migration of multi-valued paragraph fields from banner, tile_1, tile_2 and tile_3: We will use the destination plugin provided in the previously mentioned patch for Entity Reference Revisions. To use that plugin, enter 'entity_reference_revisions:paragraph' in the destination. **It is very important to use a singular paragraph, not paragraphs for the second part of the destination**
dependencies:
enforced:
module:
- paragraph_migration
id: banner
source:
plugin: csv
path: modules/custom/paragraph_migration/assets/csv/landing_pages.csv
header_row_count: 1
keys:
- id
process:
field_image:
plugin: migration
migration: files
source: banner_image
no_stub: true
field_call_to_action/title: banner_link_text
field_call_to_action/uri: banner_link
destination:
plugin: 'entity_reference_revisions:paragraph'
default_bundle: banner
migration_dependencies:
required:
- files
optional: { }
dependencies:
enforced:
module:
- paragraph_migration
id: tile_1
source:
plugin: csv
path: modules/custom/paragraph_migration/assets/csv/landing_pages.csv
header_row_count: 1
keys:
- id
- img1
process:
field_image:
plugin: migration
migration: files
source: img1
no_stub: true
field_description: desc1
destination:
plugin: 'entity_reference_revisions:paragraph'
default_bundle: tile
migration_dependencies:
required:
- files
optional: { }
dependencies:
enforced:
module:
- paragraph_migration
id: tile_2
source:
plugin: csv
path: modules/custom/paragraph_migration/assets/csv/landing_pages.csv
header_row_count: 1
keys:
- id
- img2
process:
field_image:
plugin: migration
migration: files
source: img2
no_stub: true
field_description: desc2
destination:
plugin: 'entity_reference_revisions:paragraph'
default_bundle: tile
migration_dependencies:
required:
- files
optional: { }
dependencies:
enforced:
module:
- paragraph_migration
id: tile_3
source:
plugin: csv
path: modules/custom/paragraph_migration/assets/csv/landing_pages.csv
header_row_count: 1
keys:
- id
- img3
process:
field_image:
plugin: migration
migration: files
source: img3
no_stub: true
field_description: desc3
destination:
plugin: 'entity_reference_revisions:paragraph'
default_bundle: tile
migration_dependencies:
required:
- files
optional: { }
For each tile YML migration file it is necessary to add the id and the name of the image as the id or key for the migration. This will get added to the mapping table and used by the follow-up node migration. These keys need to be unique.
- Node migration (final destination of our content): Lastly, let's review the YML file that should pull everything together from all our previous migrations into Paragaphs. There is a different approach used for multi-valued paragraphs as compared to single valued paragraphs, so pay careful attention to the details.
dependencies:
enforced:
module:
- paragraph_migration
id: landing_pages
source:
plugin: csv
path: modules/custom/paragraph_migration/assets/csv/landing_pages.csv
header_row_count: 1
keys:
- id
process:
title: title
field_banner/target_id:
-
plugin: migration
migration: banner
no_stub: true
source: id
-
plugin: extract
index:
- '0'
field_banner/target_revision_id:
-
plugin: migration
migration: banner
no_stub: true
source: id
-
plugin: extract
index:
- 1
combination_1:
plugin: get
source:
- id
- img1
combination_2:
plugin: get
source:
- id
- img2
combination_3:
plugin: get
source:
- id
- img3
combination:
plugin: get
source:
- @combination_1
- @combination_2
- @combination_3
field_tile:
-
plugin: migration
migration:
- tile_1
- tile_2
- tile_3
no_stub: true
source: @combination
-
plugin: iterator
process:
target_id: '0'
target_revision_id: '1'
destination:
plugin: 'entity:node'
default_bundle: landing_page
migration_dependencies:
required:
- banner
- tile_1
- tile_2
- tile_3
optional: { }
**field_banner/target_id y field_banner/target_id : The extract value of '0' (with single quotes) is important and isn't a mistake. If you don't do it this way, the extract process plugin will treat zero as empty and skip extracting the value. A value of 1 (without quotes) is fine for the extract plugin. It doesn't skip a numeric value of one.
**combination: This is a temporary or psuedo field that is used to collect all the source values together into the right format for later inserting into the actual destination. You can tell it is a temporary field because it has a somewhat random name that doesn't begin with the 'field_' prefix and later on it is used with a prefix of at sign (@). We use the combination field as an intermediary for the final destination of 'field_tile'.
The second part of the multi-valued paragraph migration, the iterator only accepts string values. It doesn't like numerics. The single quotes around both the '0' and the '1' are not a mistake. This is different than the earlier mention for field_banner/target_id y field_banner/target_id.
With our fully-backed migration yamls and our created content types, it is now time to see how it works.
$ drush en paragraph_migration -y
Now, run the migration:
$ drush mi --all
Yeah!!! Now we are able to migration source data into Paragraphs.
You can obtain the whole code base for the examples used in this post on GitHub.
Are you looking for help with a Drupal migration or upgrade? Regardless of the site or data complexity, MTech can help you move from a proprietary CMS or upgrade to the latest version–Drupal 8.
Write us about your project, and we’ll get back to you within 48 hours.