How to Migrate-JSON...

Posted by Juan Santovenia on February 8, 2016
Development
Migration & Upgrades
Strategy & Architecture
Support

When using the Migrate Source JSON module, it requires that all data be on a single level. To properly perform a migration through JSON, we must make changes to the data you want to migrate. The two things we should look at are id and source.

  1. id
  2. source

The id contains references or values ​​that make each record unique and source is the actual values we want to collect. Many times the id is also a source. For this example, we collect each of these values ​​as a single extracted level. Once these settings are done, you should be able to be successfully migrate data through JSON.

class GmapJSONReader extends JSONReader {
  public function getSourceFields($url) {
    $items = parent::getSourceFields($url);
    // Loop over the JSON values, walk the tree and extract as keyed values.
    foreach ($items as &$item) {
      if (isset($item['address_components']) && is_array($item['address_components'])) {
        foreach ($item['address_components'] as $component) {
          if (isset($component['long_name']) && isset($component['types']) && is_array($component['types'])) {
            foreach ($component['types'] as $type) {
              $item[$type] = $component['long_name'];
            }
          }
        }
      }
      if (isset($item['geometry']['location']['lat'])) {
        $item['lat'] = $item['geometry']['location']['lat'];
      }
      if (isset($item['geometry']['location']['lng'])) {
        $item['lng'] = $item['geometry']['location']['lng'];
      }
    }

    return $items;
  }
}
class GmapTypeJSONReader extends JSONReader {
  public function getSourceFields($url) {
    $items = parent::getSourceFields($url);
    $return = [];
    // Loop over the JSON values and extract types
    // as individual rows.
    foreach ($items as &$item) {
      if (isset($item['types']) && is_array($item['types'])) {
        foreach ($item['types'] as $type)
        // We need to use place_id as the identifier for parsing JSON
        // in the parent class.
        // But we are really wanting to import an individual type
        // only once. So over-write the place_id with the type.
        $return[] = [
          'place_id' => $type,
          'type' => $type,
        ];

      }
    }

    return $return;
  }
}
id: gmap
migration_tags: JSON
label: gmap
source:
  plugin: json_source
  path: 'https://maps.googleapis.com/maps/api/geocode/json?latlng=40.714224,-73.961452'
  headers:
    Accept: application/json
  identifier: place_id
  identifierDepth: 1
  readerClass: Drupal\d8_json_migrate\Plugin\migrate\GmapJSONReader
  fields:
    - place_id
    - types
    - street_address
    - postal_code
    - country
    - locality
    - administrative_area_level_1
process:
  type:
    plugin: default_value
    default_value: gmaps
  title: country
  field_address_type:
    plugin: migration
    migration: gmap_type
    source: types
  field_city: locality
  field_country: country
  field_postal_code: postal_code
  field_state: administrative_area_level_1
  field_street_address: street_address
destination:
  plugin: 'entity:node'
template: null
migration_dependencies: {}
migration_group: JSON
id: gmap_type
migration_tags: JSON
label: 'gmap type'
source:
  plugin: json_source
  path: 'https://maps.googleapis.com/maps/api/geocode/json?latlng=40.714224,-73.961452'
  headers:
    Accept: application/json
  identifier: place_id
  identifierDepth: 1
  readerClass: Drupal\d8_json_migrate\Plugin\migrate\GmapTypeJSONReader
  fields:
    - type
process:
  name: type
  vid:
    plugin: default_value
    default_value: address_type
destination:
  plugin: 'entity:taxonomy_term'
template: null
migration_dependencies: {  }
migration_group: JSON

 

Are you looking for help with a Drupal migration or upgrade? Regardless of the site or data complexity, MTech can help you move from a proprietary CMS or upgrade to the latest version–Drupal 8.

Write us about your project, and we’ll get back to you within 48 hours.


How to Migrate-JSON...