Mapping datasets locally [BETA]

Smart mapping and downloading files from datasets on Pennsieve

The Pennsieve platform allows users to map datasets from the platform onto your local hard disk.

Note: this is a beta feature.

To start run pennsieve map -h on your terminal to see all of the available commands

 % pennsieve map -h

  [BETA] This feature is in Beta mode and is currently still undergoing
  testing and optimization.

  The fetch command creates a folder that is associated with a
  Pennsieve dataset. Subfolders and files will be created within
  the target folder to match the dataset structure on the platfom.

  In contrast to the download command, the files will NOT be
  downloaded to the local machine, but instead, an empty file
  representing the file on Pennsieve will be created. Use the "sync"
  command to download specific files, or folders to your local
  machine.

  Using "fetch" and "sync" allows for users to efficiently map
  Pennsieve datasets to their local machines without requiring
  to download each file in the dataset. This saves space, and costs
  associated with downloading the entire dataset.

Usage:
  pennsieve map [dataset_id] [target_path] [flags]
  pennsieve map [command]

Available Commands:
  diff        List local changes compared to last fetch from Pennsieve.
  fetch       Fetch remote state to locally mapped dataset
  pull        Pull files from the server into a mapped Pennsieve Dataset.
  push        Push local changes to the remote Pennsieve Dataset

Flags:
  -h, --help   help for map

Global Flags:
      --config string   config file (default is $HOME/.pennsieve/config.ini)

Mapping a folder

To map a folder run pennsieve map <dataset node id> <directory>. This will create the file and folder structure of your dataset

Note:

<directory> must be a new folder that does not currently exist on your file system
<dataset node id> can be be found by using the commpand pennsieve dataset to show the current dataset the agent is mapped to. Otherwise you can find the datasetID by visiting app.pennsieve.io and going to the Overview Page of any dataset

Pulling mapped datasets

After mapping a dataset, run pennsieve map pull <path to the mapped file/folder>. This will begin the process of downloading your mapped data.

Note:

Please use the full file system path to your file or folder.

Other commands

The commands map diff, map fetch, map push are not currently implemented, but will be in future updates.

Updated 1 day ago