Alternative Device Profile Storage Methods

One of the several unique components of Ionic Machina is the strong notion of trust between key servers and devices. Key servers are the part of the Ionic ecosystem that holds encryption keys, and devices are the clients interacting with Ionic APIs to fetch those keys to encrypt and decrypt data. The result of this strong trust model is a slightly unusual approach to authentication in Machina Tools — Device APIs. In this post we will discuss device enrollment, device profiles, and approaches for managing device profiles that allow you to protect your most critical assets in a variety of scenarios.

DEVICE ENROLLMENT

To create and fetch data protection keys in Machina a user must have an enrolled device. Devices are enrolled in a keyspace, which is a division of the Ionic global keyspace reserved for a given set of physical key servers, which hold encryption keys. The high-level steps that occur during enrollment are as follows:

  • The user makes a request to an enrollment portal (EP) associated with a keyspace.
  • After the user authenticates, the EP returns a signed token to the device which proves that it has successfully authenticated and is allowed to access keys in the keyspace. The EP also returns an asymmetric key that the device can use to securely send data to the key server (KS).
  • The user locally creates a public/private key pair and a few additional pieces of metadata.
  • The user takes the signed token returned from the EP, the public/private key pair, and the metadata created in the previous step and makes a request to Ionic Machina asking to be enrolled. The public/private keys are encrypted so Ionic can’t read them.
  • Machina does some light validation on that request and passes the payload back to a KS associated with the keyspace.
  • The KS generates an id for the new device, creates a record for that device, creates some keys for communicating with that device, and returns information about the created device to Ionic. The keys for communicating between the device and the KS are encrypted so Ionic can’t read them.
  • Machina creates a record for the new device and returns the payload returned from the KS to the user.
  • The user unpacks the encrypted payload returned from Machina and saves the keys and metadata required to communicate securely with the KS to secure storage. This collection of metadata and keys is called the secure enrollment profile (SEP) or device profile.

As you can see, there are a lot of steps here, and this is a simplified version of the process! The important bit to keep in mind here is the end benefit of this process is the device the user has enrolled is now able to communicate securely to any KS in keyspace A using the device profile returned in the last step.

DEVICE PROFILES

The device profile is core to how devices authenticate in Machina, so it is helpful to have an idea about what is contained in a profile and how that information is used in the SDK. We’ll use Ionic’s Python SDK in an iPython shell to explore what a profile looks like. We’ll be using version 1.6.0.3 of the Python SDK. You can find documentation for the SDK at Machina Developers, along with a general guide to getting started with Machina. Note that in these examples I mask parts of some sensitive parameters with XXXXX.

Exploring the agent

In [1]: import ionicsdk
In [2]: a = ionicsdk.Agent()
In [3]: profile = a.getactiveprofile()
In [4]: profile
Out[4]: <DeviceProfile name="devx" deviceid="HVzG.4.62b62a59-6cc1-XXXX-XXXX-XXXXXXXXXXXX" keyspace="HVzG">
In [5]: profile?
Type:           DeviceProfile
String form:    <DeviceProfile name="devx" deviceid="HVzG.4.62b62a59-6cc1-XXXX-XXXX-XXXXXXXXXXXX" keyspace="HVzG">
File:           /anaconda2/envs/ionic-examples/lib/python3.6/site-packages/ionicsdk/common.py
Docstring:
!Data class for storing device profile information (a.k.a. SEP data).
 
This class stores device profile information which is the result of
a successful device registration performed via Agent.createdevice().
 
A device profile is also known as a SEP (Secure Enrollment Profile).
 
Instance Variables:
 
    name (string): The human readable name associated with this
                   profile.
 
    deviceid (string): Device ID generated by Ionic.com during
                       registration that is performed by calling
                       Agent.createdevice().
 
    keyspace (string): The device profile key space.
 
    server (string): The Ionic.com server associated with this
                     device profile.
 
    creationtimestampsecs (int): The time at which this profile was
                       created in UTC seconds since January 1, 1970.
 
    aesCdIdcProfileKey (bytes): The private AES key shared between
                                client and Ionic.com.
 
    aesCdEiProfileKey (bytes): The private AES key shared between
                               client and EI (Enterprise
                               Infrastructure).
Init docstring:
!Initializes the agent config object with provided inputs.
 
@param
    name (string): The human readable name associated with this profile.
@param
    deviceid (string): Device ID generated by Ionic.com during registration.
@param
    keyspace (string): The device profile key space.
@param
    server (string): The Ionic.com server associated with this device profile.
@param
    creationtimestampsecs (int): The time at which this profile
               was created in UTC seconds since January 1, 1970.
@param
    aesCdIdcProfileKey (bytes): The private AES key shared
                                between client and Ionic.com.
@param
    aesCdEiProfileKey (bytes): The private AES key shared
              between client and EI (Enterprise Infrastructure).

Let’s explain what just happened there.

First we imported the ionicsdk python package and created an Agent object. Then we grabbed the active profile and printed out some information about it. Note that you must have already enrolled for this step to work.

From the details we can see that the active device profile is associated with keyspace “HVzG”. Asking for more details about the profile object, we see it is composed of seven fields:

  1. A human readable name, which can help users keep track of which devices are used for different use cases.
  2. A device id, which uniquely globally identifies the device.
  3. A keyspace, which tells us which keyspace the profile allows us to create / access keys with.
  4. A server, which tells us the url that we should make requests to. This is always Ionic.com unless you are working with an on-premises installation of Ionic Machina.
  5. The time the device was created.
  6. The aesCdIdcProfileKey, a key used to securely communicate with Ionic, the service routed to by the url defined in server.
  7. The aesCdEiProfileKey, a key used to securely talk to key servers in the keyspace defined in keyspace.

As you can probably guess, two of these fields are quite important to keep safe!

In Machina Tools — Python SDK, the responsibility for saving and loading device profile data is handled by a persistor. If we look at the code for the python agent (you can find it somewhere similar to lib/python3.6/site-packages/ionicsdk/agent.py in your python installation), we can see the following in the initialization code for the ionicsdk.Agent object.

Agent Initialization

class Agent(AgentKeyServicesBase):
    """Agent class performs all client/server communication with
    Ionic.com.
    """
    def __init__(self, agentconfig = None, profilepersistor = None, loadprofiles = True):
        """An Agent object can be initialized with different profile
        persistors and a config object, or defaults
 
        By default an Agent will load profiles from the platform-
        specific device profile persistor (if one exists for the
        platform, otherwise an AGENT_NO_PROFILE_PERSISTOR IonicException
        may be thrown if no other persistor is specified). Loading can
        be disabled, or a DeviceProfilePersistorPlaintextFile,
        DeviceProfilePersistorPasswordFile, or
        DeviceProfilePersistorAesGcmFile can be used to load profiles
        from a file.
 
        An AgentConfig object or configuration file can specify network
        communication parameters.
 
        Args:
            agentconfig (AgentConfig, optional): An AgentConfig object
                can specify network configuration parameters.
            profilepersistor (DeviceProfilePersistorBase, optional): A
                device profile persistor. If unspecified or None, will
                use platform default which will be shared across all
                ionic-enabled applications. Subclasses are available to
                load profiles from plaintext, passworded, or encrypted
                files.
            loadprofiles (bool, optional): Set to false to prevent
                loading of profiles. Profiles must be added or loaded
                using Agent.*profile[s] methods before using the Agent
                object for network communication.
        """
        self._cAgent = None
 
        cAgentConfig = AgentConfig._marshalToC(agentconfig)
 
        if loadprofiles:
            pp = None
            if profilepersistor:
                if not isinstance(profilepersistor, DeviceProfilePersistorBase):
                    raise Exception('profilepersistor must be an instance of DeviceProfilePersistorBase')
                pp = profilepersistor._cPersistor
            self._cAgent = _private.cLib.ionic_agent_create(pp, cAgentConfig)
        else:
            self._cAgent = _private.cLib.ionic_agent_create_without_profiles(cAgentConfig)
 
        if not self._cAgent:
            raise IonicException('Failed to create an Agent object. Check the log to know the reason for this error (common cause is failure to initialize internally).', IonicError.AGENT_ERROR)
        _private.cLib.ionic_agent_set_metadata(self._cAgent, _private.CMarshalUtil.stringToC("ionic-agent"), _private.CMarshalUtil.stringToC("IonicSDK/1 Python/" + _private.IONIC_SDK_VERSION))

We can see that the agent uses the profilepersistor passed in (or the default persistor if nothing is passed) to load a profile.

Machina Tools — SDK comes with a set of persistors build in. You can see general documentation about those here, and you can see the documentation for the Python-specific implementations here.

  • DeviceProfilePersistorPlaintextFile
    • The profile information is written to an plaintext JSON file.
  • DeviceProfilePersistorDefault
    • The implementation depends on the platform. On Mac this uses the Apple KeyChain, and on Windows this uses the DPAPI. There is no default implementation for some other platforms, e.g. Linux.
  • DeviceProfilePersistorAesGcmFile
    • The profile information is written to an encrypted JSON file, which is protected via AES-GCM with a key and auth data that you manage.
  • DeviceProfilePersistorPasswordFile
    • The profile information is written to an encrypted JSON file, which is protected via a password that you manage.
  • SecretShareProfilePersistor
    • The profile information is written to a file protected with a set of secrets, where a minimal set of those secrets must be provided to unlock the file. This uses Shamir’s Secret Sharing to protect data.

Let’s try out the plain text persistor so we can get a look at what a device profile looks like when it stored on disk.

Python SDK plain text export

import ionicsdk
import os
 
# Create default agent
a = ionicsdk.Agent()
 
# Set up the plaintext persistor
a2_persistor = ionicsdk.common.DeviceProfilePersistorPlaintextFile("exampleprofile.pt")
 
# Create a new agent, add 1 profile, and save.
a2 = ionicsdk.Agent(loadprofiles=False)
a2.addprofile(a.getactiveprofile())
a2.saveprofiles(a2_persistor)

Note that we could also do the same thing using the ionic-profiles tool.

ionic-tools profile export

timothy$ ./ionic-profiles convert --target-persistor plaintext --target-persistor-path exampleprofile.pt --device-id HVzG.4.62b62a59-6cc1-XXXX-XXXX-XXXXXXXXXXXX
---> Initializing Ionic Agent Default Persistor profiles
---> Loading profiles in 'default' Persistor in ''
---> Initializing Ionic Agent Plaintext Persistor profiles
Converting profile with ID 'HVzG.4.62b62a59-6cc1-XXXX-XXXX-XXXXXXXXXXXX' to 'plaintext' Persistor in 'exampleprofile.pt'
[SUCCESS] Converted profile(s) to plaintext in file: exampleprofile.pt

Now that we have the file saved on disk, we can check the contents.

Check pt persistor contents

timothy$ cat exampleprofile.pt | python -m json.tool
{
    "activeDeviceId": "",
    "profiles": [
        {
            "aesCdEiKey": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
            "aesCdIdcKey": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
            "creationTimestamp": 1530653969,
            "deviceId": "HVzG.4.62b62a59-6cc1-XXXX-XXXX-XXXXXXXXXXXX",
            "name": "devx",
            "server": "https://dev-api.ionic.com"
        }
    ]
}

The structure of the file is quite simple. The AES keys are hex-encoded 256 bit keys. The timestamp is a UNIX timestamp (seconds since the epoch). The device id, name, and server fields are all strings. The list of profiles is nested in a JSON wrapper object which also has a field to mark the currently active device id.

ALTERNATIVE DEVICE PROFILE STORAGE

You may find yourself in a situation where none of these methods for storing profile data are appropriate, or they may just be unwieldy. For example, if you are running ephemeral compute resources like AWS Lambda or Azure Functions you may not have a disk to read and write profile data to!

In that case, one path forward is be to implement a custom persistence method for device profiles. We’ll start with a minimal implementation, again using the Python SDK.

Create agent with single profile from custom source

import binascii
import json
import ionicsdk
 
# You start with a JSON string, which is the profile definition (the section inside the profiles array for the plain text persistor)
json_profile = "..."
profile_dict = json.loads(json_profile)
profile = ionicsdk.DeviceProfile(**{
   name=d['name'],
   deviceid=d['deviceId'],
   keyspace=d['deviceId'].split(".")[0],
   server=d['server'],
   creationtimestampsecs=int(d['creationTimestamp']),
   aesCdIdcProfileKey=binascii.unhexlify(d['aesCdIdcKey']),
   aesCdEiProfileKey=binascii.unhexlify(d['aesCdEiKey']),
})
 
# Create an agent object
a = ionicsdk.Agent(loadprofiles=False)
a.addprofile(profile)
a.setactiveprofile(profile)
 
# Agent is ready to use

In this example we start with the profile data stored in a JSON string and we create a ionicsdk.DeviceProfile object from the contents of that string. We then create an Agentobject with no profiles loaded, and we call the addprofile and setactiveprofile methods on the Agent to get it configured with the JSON profile data.

This is not too bad, but it’s not something we want to have to repeat in all our applications, especially when we recognize we didn’t even consider the logic for grabbing the JSON string. The default behavior of the Ionic Agent is to load profiles from its configured source immediately upon creation, and we’d like to be able to have similar behavior regardless of where we decide to store our device profile record. Writing a custom persistor would be a good next step to clean up this code.

However, if we look at how the persistor object which is passed into Agent.__init__ is used in the code above, we see that the function expects the persistor to have a _cPersistor property. Instead of trying to get our persistor to match the Ionic C interface, we can make a small modification to our approach and keep our profile handling logic in pure Python. Instead of subclassing the DeviceProfilePersistorBase to create a new persistor, we will subclass Agent so we can override its __init__ method. That looks like this:

https://gist.github.com/turtlemonvh/229cd5ffaa5b9486e481e233519c4863

That looks like a lot of code, but much of it is boilerplate for clean error handling. The important bit is the loadprofiles and saveprofiles functions, and the calls to loadprofiles in the Agent.__init__ method. From that you can see that all loadprofiles does is call a function on the persistor which returns a ionicsdk.DeviceProfileobject, and then calls addprofile on the agent object to add that device profile to the agent. An implementation of the the custom persistor that extends the behavior of DeviceProfilePersistorPlaintextFile to handle the creation of the directory path to the plain text file if it doesn’t exist is included to demonstrate how to use these interfaces to save and load device profiles.

STORING DEVICE PROFILES IN SECRETS VAULTS

From the previous sections we can see that, at a high level, we can think of a device profile as simply a set of credentials that can be used to authenticate to Machina. Further, we saw that if we can save and load the string containing those credentials securely, we have what we need to interact with Ionic.

Thankfully there has been a lot of great work put toward figuring out how to securely provide applications with the credentials they need to operate. Whether those credentials are in the form or usernames and passwords, API keys, OAuth tokens, public/private key pairs, or other formats, the solutions are similar. The approaches taken often depend on the sensitivity of the resources protected by a set of credentials. For some applications, keeping credentials in plain text on disk can be good enough. For example, this is often how credentials to cloud providers like AWS are managed. Password and key protected files are also quite common, which is why these options are available out-of-the-box in Ionic SDKs.

A pattern that is a bit more common for sensitive application workloads is to use a “secrets vault,” an application that provides mechanisms for other applications to make requests to fetch credentials at run time. Some popular secrets vaults include Hashicorp VaultSquare KeyWhizNike CerberusCyberArk Conjur, and Thycotic Secret Server. A nice recent overview of the field can be found in this Github gist. In addition to these systems, some platforms have secret storage as a first class feature, for example Docker Swarm and Kubernetes. One of these systems that we use for some operations at Ionic is AWS Parameter Store. AWS Parameter Store doesn’t market itself as a password vault, but since it allows users to store small snippets for configuration protected with KMS keysit has become a go-to system for managing sensitive credentials in the AWS ecosystem, especially when combined with other AWS capabilities like EC2 IAM Roles.

Building on the device profile loading code we developed in the previous section, we can now put together an implementation of a custom persistor that reads configuration from AWS Parameter store.

https://gist.github.com/turtlemonvh/50318adb7b5f1df65a164d82200d7862

There are a few interesting details in this code. First, we don’t implement the saveprofiles functionality. The idea here is that the profile is “read only” from the perspective of the application. The device enrollment can be performed on completely different physical device from the device which will use the profile at run time, and the secret will be loaded into AWS Parameter Store before the application starts. One way to do this would be to:

  1. Create a new device profile by enrolling using Ionic tools
  2. Save that profile to a plain text
  3. Grab the JSON describing the profile and save it into AWS Parameter Store via the web console

Second, most of the logic is in the loadprofiles function, and this logic is specific to AWS parameter store. There isn’t much Ionic-specific logic to worry about.

There are some nice benefits to storing secrets in a secret vault, as we have shown here. We can treat device profiles similar to any other application secret, which means we can use existing controls and audit capabilities to determine how these secrets are used. We can also rotate these credentials to minimize the impact of credential leak. It is also simple to handle ephemeral compute resources, for example serverless workloads like AWS Lambda.

SUMMARY

In this post, we discussed device profiles. First we covered their creation and their use in securing communications with various components of the Ionic platform. Then we went deep into how these profiles can be treated similar to any other set of application credentials and how that treatment allows us to use modern tooling like application secrets vaults to secure the authentication of Machina-protected workloads.

We hope this post has been helpful to furthering your understanding of the Ionic platform and how you can successfully integrate Machina to protect your most valuable digital assets. Please reach out to us with any feedback or requests for future topics.