Uploading data using CLI

How do I upload files and folders to the platform using the command line interface (CLI)?

You can upload files to the Pennsieve platform programmatically using the Pennsieve Command Line Interface (CLI). Below are instructions to:

  • Install the CLI
  • Setup a profile for your account
  • Upload data using the CLI

Installation

In order to use the CLI, you must install the Pennsieve CLI Agent. In addition, you need to create an API token & secret in the web application.

Create connection profile

Once the CLI is installed, the next step is to create a profile that allows the CLI to access the platform on your behalf. To create a profile, run pennsieve_agent profile create from the command line:

`pennsieve_agent profile create`

You then be taken through the steps to name your profile. Give your profile a unique name, enter the newly created API token, and type y to save the profile:

$ pennsieve_agent profile create

```Create a new profile:
  Profile name: [default] my-profile
  API token: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
  API secret: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Creating new profile: 'my-profile'
Continue and write changes? (y/n) y```

To verify that your profile was set up correctly, run pennsieve_agent whoami

$ pennsieve_agent whoami

```+-----------------+----------------------------------------------+
| NAME            | [email protected]                                | 
| USER ID         | N:user:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx  | 
| ORGANIZATION    | Your Organization                            | 
| ORGANIZATION ID | N:organization:xxxxxxxx-xxxx-xxxx-xxxx-\ .   |
+-----------------+----------------------------------------------+```

Uploading data using the CLI

Once you have created a profile associated with your Pennsieve account, you can upload data to a dataset. To select which dataset you want to upload data to, use the pennsieve_agent datasets command to see a list of all available datasets that you can access:

$ pennsieve_agent datasets

+-----------------------------------+---------------+------------+
    | DATASET                           | NAME          | STATUS     |
    +-----------------------------------+---------------+------------+
    | N:dataset:xxx-xxx                 | My dataset 1  | No Status  |
    | N:dataset:xxx-xxx                 | My dataset 2  | No Status  |
    | N:dataset:xxx-xxx                 | My dataset 3  | No Status  |
    +-----------------------------------+---------------+------------+

Set which dataset you want to upload to using the pennsieve_agent use command. You can pass the dataset name or the dataset id as the argument of the command:

`$ pennsieve_agent use N:dataset:98a1fcd-9346-49cc-bdf1-52c11faa060` 

or

`$ pennsieve_agent use "My dataset 1"` 

Finally, use the pennsieve_agent upload command to upload your data. This can point to a file or a folder on your computer

`$ pennsieve_agent upload /path/to/my/file.txt` 

To upload a deeply nested folder, pass the --recursive flag:

`$ pennsieve_agent upload --recursive /path/to/my/folder` 

If you want to upload data to a specific collection/folder inside the dataset, you can specify this with the --folder parameter. You can find the collection id in the url for web application when you navigate into the specific folder in the files browser. For example:

`$ pennsieve_agent upload --folder=N:collection:d4cb103e-7765-4... ~/Desktop/data/file.txt` 

For more information about the pennsieve_agent upload, you can use the --help flag:

`$ pennsieve_agent upload --help` 

To see all available functionality in the CLI, use the help command:

`$ pennsieve_agent help`

NOTE: See the Developers Documentation for additional information about the functionality of the CLI.


Did this page help you?