Jack Whiting

2021/03/22

Deep Dive: Building a Shopify Add-on for Statamic

Statamic

Table of Contents

  1. Background
  2. The Admin API
  3. Setting Up Access
  4. Fetching The Products
  5. Saving the Products
  6. Fetching the Variants
  7. Fetching The Images
  8. Interacting With The Data
  9. Handling Product Changes
  10. Integrating Webhooks
  11. Handling Stock Updates
  12. Handling Deleted Products
  13. Wrapping Up

You may have seen it already, or not, but I recently launched a Shopify add-on for Statamic.

I wanted to breakdown a bit of the choices made when building the add-on as well as how I integrated Shopify with Statamic.

Hope you enjoy!

Background

I recently came into contact with a client who was looking at building a store for a rice company. They wanted to sell their products through Shopify but wanted a way to focus the site on content; recipes, careers, blogs, etc.

Shopify is great at doing what it does, e-commerce, however, it's plainly pitiful in handling anything outside of that. Sure, there are several add-ons that try to solve this - but they are the equivalent of band-aids on a wound.

They wanted something focused on content, relations between content, and something easy for the client to manage. Enter Statamic. It's absolutely awesome as a CMS, powerful out of the box, and easily extendable as it's built on top of Laravel.

I started playing around with the idea of how I could integrate the two. Shopify has two very extensive APIs - the Admin API and the Storefront API. After browsing the docs, and a little playing around, I decided the best way to go forward would be to integrate Shopify into the CMS.

The integration was not looking to recreate what Shopify does with managing customers, marketing, etc. Shopify is great at this, so I wanted to leave it to do its thing. The client wanted to primarily extend the content provided on their products with ingredients, key features, related recipes, etc.

I needed to find a way to get the product data into Statamic.

Enter the Admin API.

The Admin API

The Admin API gives us access to a lot of data. It is not all needed in this integration, I mainly wanted access to:

  • Products
  • Variants
  • Images

Laravel has a great way of dealing with http requests. I could quite easily use this to ping the URLs and handle the requests, however, I wanted to integrate something a bit more battle-tested under the hood.

I had a look around at available solutions and how people integrated code. I then found the great library php-shopify on GitHub.

Setting Up Access

I didn't want to keep having to pass through the configuration values - DRYer code is better. In our ServiceProvider, we create the config and initialise it to use throughout the add-on.

private function setShopifyApiConfig(): void
{
    $config = [
        'ShopUrl' => config('shopify.url'),
        'ApiKey' => config('shopify.auth_key'),
        'Password' => config('shopify.auth_password'),
    ];

    ShopifySDK::config($config);
}

 

There are a few options wrapped up here to ensure that if a user wants to caches their config then things wouldn't break. These values are all pulled from the `env` file in the config.

Fetching The Products

When fetching products there are a few patterns a user can fit into:

  • They could have a small subset with limited variations.
  • The could have a large set of data with no real configurations.
  • The could have both a large set of data with a lot of configurations.

To ensure that the add-on could handle all of these configurations, I set up a way to pull in the paginated resources and dispatch them to the queue so that they can be worked on at their own pace.

The Shopify Admin API limits you to getting 250 products at once from the products endpoint.

When first integrating the solution, I bumped into a few issues with the paginated results. Fortunately, with a few stumbling, a bit of fiddling, and some lucky searches. I came to the following solution.

$shopify = new ShopifySDK;

$productResource = $shopify->Product();
$products = $productResource->get(['limit' => config('shopify.api_limit')]);
$next_page = $productResource->getNextPageParams();

// Initial Loop
ImportAllProductsJob::dispatch($products);

// Recursively loop.
while ($next_page) {
    $products = $productResource->get($productResource->getNextPageParams());
    $next_page = $productResource->getNextPageParams();

    ImportAllProductsJob::dispatch($products);
}

Shopify passes the pagination data through in the header. What this code does is fetches the data, checks if there are any headers, and then dispatches them to the job to handle the import. It then repeats this until the $next_page var comes back as null.

Saving the Products

Statamic is primarily a flat-file CMS and I didn't want to deviate away from this. Fortunately, I didn't need to.

Statamic provides a Entry facade, which allows us to fetch, update, and insert data. This works whether our data source is flat files or a database and made it great to expand upon.

In our ImportJob, we first check if the user has imported the product before and if not we instantiate it. This was a rather simple fetch and check if null.

$entry = Entry::query()
	        ->where('collection', 'products')
          ->where('slug', $this->slug)
          ->first();

if (!$entry) {
	$entry = Entry::make()
		        ->collection('products')
	            ->slug($this->data['handle']);
}

 

This logic is used in both products and variants.

Next, to handle data coming from Shopify, I wanted to make sure that a user could set their preference on how they wanted the data to be pulled in. The user could have the following scenarios:

  1. Did they want Shopify to be the source of truth and overwrite the data in our CMS?
  2. Do we want the content edited in the CMS to be the source of truth and ignore any incoming overwrites?
  3. Does the user want Shopify to overwrite some things and not others?

Pulling the main bulk of data for a product is relatively simple. I wrapped the main fields that a user could overwrite with a check for if the config values are set as true. If the config is set to overwrite, we use the Shopify data, if not we use the local data.

If the entry doesn't exist we always use the Shopify data.

For the actual data pulled, the add-on fetches the ID, date, content, vendor information, types, tags, variants, and images.

$data = [
    'product_id' => $this->data['id'],
    'published_at' => Carbon::parse($this->data['published_at'])->format('Y-m-d H:i:s'),
    'title' => (!$entry || config('shopify.overwrite.title')) ? $this->data['title'] : $entry->title,
    'content' => (!$entry || config('shopify.overwrite.content')) ? $this->data['body_html'] : $entry->content,
    'vendor' => (!$entry || config('shopify.overwrite.vendor')) ? $this->formatStrings($this->data['vendor']) : $entry->vendor,
    'type' => (!$entry || config('shopify.overwrite.type')) ? $this->formatStrings($this->data['product_type']) : $entry->type,
    'tags' => (!$entry || config('shopify.overwrite.tags')) ? $formatTags : $entry->tags,
];

Fetching the Variants

Variants were a little more complex. Each product can have, well a s!*t-tonne, of variants.

In the case of my client they didn't, but I wanted to make this versatile for any data you could throw at it. Alongside this, variants are the source of truth for the stock quantity and whether the product should be sold if it runs out of stock.

You could pull these in with JavaScript on the front-end, but I wanted to get these into Statamic so that a user could adapt with additional data with any specifics to the variant.

When using the products on the front-end some of this logic will be used to check if a product is in stock or not.

$entry->data([
    'variant_id' => $variant['id'],
    'product_slug' => $product_slug,
    'title' => $variant['title'],
    'inventory_quantity' => $variant['inventory_quantity'],
    'inventory_policy' => $variant['inventory_policy'],
    'price' => $variant['price'],
    'sku' => $variant['sku'],
    'grams' => $variant['grams'],
    'requires_shipping' => $variant['requires_shipping'],
    'option1' => $variant['option1'],
    'option2' => $variant['option2'],
    'option3' => $variant['option3'],
    'storefront_id' => base64_encode($variant['admin_graphql_api_id']),
])->save();

To identify the variants against the product we add the slug to each variant - we use this in a custom field for the front-end that shows the variants on the product and lets you edit them in the stack editor.

The problem with the Admin API (back-end) and the Storefront API (front-end) is that they require different IDs to interact with the data. This gave me a little bit of a headache at first but turns out - all that needed to be done was running a base64_encode on the admin_graphql_api_id that was returned in the Admin API.

Fetching The Images

Sweet, at this point I was getting there with a lot of the data I wanted to pull in. However, I needed the images which, again, caused a few problems of their own.

Firstly, Images came with data appended that we didn't want. So we had to clean this up.

$cleanUrl = strtok($url, '?');
$imageName = substr($url, strrpos($url, '/') + 1);

Secondly, we had to find a way to download the image from the Shopify CDN and store it locally, or on S3, or any other file system. To grab the download we use the Storage facade and the UploadedFile class. This lets us temporarily store the image.

Storage::disk('local')->put($imageName, file_get_contents($cleanUrl));

$file = new UploadedFile(realpath(storage_path("app/$name")), $name);

Thirdly, we needed to pass the image to the Statamic Asset container so it would appear in the admin, could be distributed to any filesystem the user has set up and could be utilised in any way the user wanted.

This was handled by checking if the Image already existed (we don't need to double save) and then passing the temp file we created and uploading it.

// Check if it exists first - no point double importing.
$asset = Asset::query()
    ->where('container', 'shopify')
    ->where('path', 'Shopify/' . $name)
    ->first();

if ($asset) {
    return $asset->hydrate();
}

// If it doesn't exists, let's make it exist.
$path = Path::assemble('Shopify/', $file->getClientOriginalName());

$asset = Asset::make()
    ->container('shopify')
    ->path($path);

$asset->upload($file)->save();
$asset->hydrate();

Finally, we had to clean up the temporary file we created as we didn't want to clog things up.

Storage::disk('local')->delete($name);

Interacting With The Data

Great, we had most of the data stored in Statamic, but we didn't really have a great way to interact with it. Most of the fields utilised were standard text fields, content, and images.

This was great, but I wanted to see the variants inside of a product as they all have very unique IDs that a user would be confused by. To do this I took a bit of inspiration from the great commerce add-on Statamic Butik.

Jonas sets up a way to grab the variants per product and passes them to a Vue component. His variants are set in stone and defined in PHP however I was looking to give the user complete control.

Firstly, I set up a VariantBlueprint class which extends the Blueprint facade.

class VariantBlueprint extends Blueprint
{
    public function __invoke()
    {
        $bp = Blueprint::find('collections/variants/variant');

        if (! $bp) {
            return 'No variant data found';
        }

        return $bp;
    }
}

What this does is searches the filesystem for the variant YAML file and then interprets it and returns it.

I then set up a new FieldType named Variants.

public function preload()
{
    $product = $this->field()->parent();

    if (!$product->initialPath()) return;

    $variantBlueprint = new VariantBlueprint();
    $variantFields = $variantBlueprint()->fields()->addValues([])->preProcess();

    return [
        'action'             => cp_route('shopify.variants.store'),
        'variantIndexRoute'  => cp_route('shopify.variants.index', $product->slug()),
        'variantManageRoute' => cp_route('shopify.variants.store'),
        'variantBlueprint'   => $variantBlueprint()->toPublishArray(),
        'variantValues'      => $variantFields->values(),
        'variantMeta'        => $variantFields->meta(),
        'productSlug'        => $product->slug(),
    ];
}

Here, we fetch the parent product. If there is no parent we return void. This allows us to check if we are on a previously saved product or a new product that wouldn't have any variants.

We then grab the blueprint, process the fields, and then pass this along with the routes we need to edit any data.

This all is loaded into a Vue component that triggers the Stack to open and allows us to update the variant data.

Boom!

Handling Product Changes

After a bit of testing, I realised that the data was only updating when the user ran the importer. This could cause problems of stock being out of data or even having products deleted on Shopify and being listed on the CMS.

To handle this I looked at integrating Webhooks.

Integrating Webhooks

When you set up Shopify webhooks, the data being sent back contains a header that we need to verify against a secret.

To do this, I created a trait that could be integrated into any current, and future, webhooks.

protected function verify($data, $hmac_header): bool
{
    $calculated_hmac = base64_encode(hash_hmac('sha256', $data, config('shopify.webhook_secret'), true));

    return hash_equals($hmac_header, $calculated_hmac);
}

This calculates an HMAC based on the secret and data we want to check. It then verifies this against the HMAC that Shopify passes in the header.

Handling Stock Updates

If a product only has one stock left, and we don't update our products in Statamic on a regular basis, we might still be showing the products to the user as available - thus causing drop-offs in cart conversions.

Shopify has a webhook for when an order is placed. This seemed like the perfect point to listen out and update the stock of any products.

In our add-on, we have an endpoint for this, which we take back the data, process it against our verify function and then decode the data returned.

public function listen(Request $request)
{
    $hmac_header = $request->header('X-Shopify-Hmac-Sha256');
    $data = $request->getContent();
    $verified = $this->verify($data, $hmac_header);

    if (! $verified) {
        return response()->json(['error' => true], 403);
    }

    // Decode data
    $data = json_decode($data);

    // Fetch Single Product
    $shopify = new ShopifySDK;

    foreach ($data->line_items as $item) {
        $product = $shopify->Product($item->product_id)->get();
        ImportSingleProductJob::dispatch($product);
    }

    return response()->json([], 200);
}

If successful, the endpoint contains a breakdown of all the line_items purchased. We then trigger a single fetch on each product to update any data we need to.

This webhook is completely optional. If you aren't interested in stock or want to do handle this in other ways - you don't want to use it.

Handling Deleted Products

We do a similar thing when products are deleted, we check the webhooks integrity against our verify the request and then delete the product that is returned.

Wrapping Up

Hope you enjoyed this dive into the back-end of the Shopify add-on along with why a few decisions were made and how they were achieved. I'm hoping to get around to doing a dive in the front-end soon - look out for it!

If you want to find out more or ask me any questions, please reach out to me on @jackabox.