This guide explains on how to use the Handle API using the bash shell and scripts to perform common tasks.

Use the right port!

Always make sure to use the right port! Test prefixes use port 8003 and production prefixes use ports 8000-8007 (your exact port number will be provided by the SURF service desk).

Setup

Setting up curl for Mac OS users

Curl is used to interface with the API. Mac OS does not always come with the right curl version nor is it compiled by default with OpenSSL support.

If you are using the wrong version of curl you will see a message similar to the one below:

* Trying x.x.x.x...
* TCP_NODELAY set * Connected to epic-pid.storage.surfsara.nl (x.x.x.x) port 8007 (#0)
* WARNING: SSL: CURLOPT_SSLKEY is ignored by Secure Transport. The private key must be in the Keychain.
* WARNING: SSL: Certificate type not set, assuming PKCS#12 format.
* SSL: Can't load the certificate "/<path>/<cert>.pem" and its private key: OSStatus -25299
* Closing connection 0
curl: (58) SSL: Can't load the certificate "/<path>/<cert>.pem" and its private key: OSStatus -25299

To compile the correct version of curl:

Use homebrew to download a pre-compiled curl with OpenSSL enabled:
```
brew install curl
```
Get the location where curl was installed and check that is has OpenSSL support:
```
brew info curl
```

Make this version the default by pointing the OS $PATH to the newly installed version:

echo 'export PATH="/opt/homebrew/opt/curl/bin:$PATH"' >> ~/.zshrc

Sessions

Handles can be created one by one using independent API calls. However, if you need to complete more then one API call in a sitting, we recommend initiating a session first. A session only requires authorisation once, at the beginning of the session,instead of after every API call. Sessions will timeout after 10 minutes of inactivity.

Close your session

Always make sure to close a session after all calls have been completed.

Example bash scripts using curl

Create a handle

The following shows a simple script to create a handle for a given URL. You will need to set your authentication information, port, URL for the PID and any other information within "<>" brackets.

Note that the within the request, under index '100' the type is always set as 'HS_ADMIN', the handle key as '0.NA/$PREFIX' and the index key as '200'. The script also send two lines in the header of the request to indicate the JSON format of the sent data and the certificate used for authentication.

#!/bin/bash

#### modify following lines ####
# Path to server
PORT=<port>
PID_SERVER=https://epic-pid.storage.surfsara.nl:${PORT}/api/handles

# PID
URL="<url>"
PREFIX="<prefix>"
SUFFIX=$(openssl rand -base64 60 | tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1) #generates random alphanumeric string of length 32

# Authentication 
# assuming <INDEX>:<PREFIX>/USER01 for authentication
INDEX=<index>
USER="USER01"
MY_PATH="<path to folder with key/certificate>"
PRIVKEY=${MY_PATH}/${PREFIX}_${USER}_${INDEX}_privkey.pem
CERTIFICATE=${MY_PATH}/${PREFIX}_${USER}_${INDEX}_certificate_only.pem
#### end modify lines ####

DATA=`cat <<EOF
{"values":[\
{"index":1,"type":"URL","data":{"format":"string","value":"$URL"}},\
{"index":100,"type":"HS_ADMIN","data":{"format":"admin",\
"value":{"handle":"0.NA/$PREFIX","index":200,"permissions":"011111110011"}}}\
]}
EOF
`

echo $DATA

curl -v -k --key $PRIVKEY --cert $CERTIFICATE \
    -H "Content-Type: application/json" \
    -H 'Authorization: Handle clientCert="true"' \
    -X PUT \
    --data $DATA \
    $PID_SERVER/$PREFIX/$SUFFIX

Show handle contents

Do not use the API for foreign handles

To prevent malicious actors from harming our service, we have a security feature that bans IP addresses that attempt to resolve prefixes that we do not host using our API after 3 attempts. The first ban will only last for 10 minutes but each successive failure will double the ban time.

  If you are testing new API calls and scripts, we recommend doing so using our test environment as it is not subject to this security policy (see here for more information).   

If you are attempting to resolve a handle that doesn’t belong to you, we recommend doing so with the global resolver and/or your web browser (see here for an example).

  If you are unsure as to what your prefix is, please create a ticket in the service portal and we’d be happy to remind you.

In this example the created handles metadata can be retrieved. Remember to replace any fields with "<>" brackets with your own values.

#!/bin/bash
 
#### modify following lines ####
# Path to server
PORT=<port>
PID_SERVER=https://epic-pid.storage.surfsara.nl:${PORT}/api/handles
 
# PID
PREFIX="<prefix>"
SUFFIX="<suffix>"
#### end modify lines ####
 
curl $PID_SERVER/$PREFIX/$SUFFIX

Modify a handle

Modifying a handle follows much the same process as creating a new handle. You can send a request using curl to modify the reference location of the handle (by modifying the value at index 1) or modify/add metadata values at any other indices (except for 1 and 100). The following script specifically adds the value 'some metadata value' with type ‘SOMETYPE’ at index 2 and 3. Note the addition of the parameter 'index=various' to the URL. Remember to replace any fields with "<>" brackets with your own values.

#!/bin/bash

#### modify following lines ####
# Path to server
PORT=<port>
PID_SERVER=https://epic-pid.storage.surfsara.nl:${PORT}/api/handles

# PID
PREFIX="<prefix>" 
SUFFIX="<suffix>"

# Authentication 
# assuming <INDEX>:<PREFIX>/USER01 for authentication
INDEX=<index>
USER="USER01"
MY_PATH="<path to folder with key/certificate>"
PRIVKEY=${MY_PATH}/${PREFIX}_${USER}_${INDEX}_privkey.pem
CERTIFICATE=${MY_PATH}/${PREFIX}_${USER}_${INDEX}_certificate_only.pem
#### end modify lines ####
 
curl -v -k --key $PRIVKEY --cert $CERTIFICATE \
    -H "Content-Type: application/json" \
    -H 'Authorization: Handle clientCert="true"' \
    -X PUT --data \
        '{"values": [
            {"index":2,"type":"SOMETYPE","data":{"format":"string","value":"some metadata value"}},
            {"index":3,"type":"OTHERTYPE","data":{"format":"string","value":"other metadata value"}}
        ]}' \
    $PID_SERVER/$PREFIX/$SUFFIX?index=various

Create multiple handles in a session

The session functionality allows the creation, modification, and deletion of multiple handles within one session and authentication call.

Close your session

Always make sure to close a session after all calls have been completed.

The following script creates 3 handles within one session. Remember to replace any fields with "<>" brackets with your own values.

#!/bin/bash

#### modify following lines ####
# Path to server
PORT=<port>
PID_SERVER=https://epic-pid.storage.surfsara.nl:${PORT}/api/handles

# PID
PREFIX="<prefix>" 

# Authentication 
# assuming <INDEX>:<PREFIX>/USER01 for authentication
INDEX=<index>
USER="USER01"
MY_PATH="<path to folder with key/certificate>"
PRIVKEY=${MY_PATH}/${PREFIX}_${USER}_${INDEX}_privkey.pem
CERTIFICATE=${MY_PATH}/${PREFIX}_${USER}_${INDEX}_certificate_only.pem
#### end modify lines ####

# create a session and capture the session id
echo "creating a session"
session_resp=$(curl -s -k --url ${PID_SERVER}/api/sessions -X POST)
echo "creating a session response is: ${session_resp}"
session_id=$(echo $session_resp | awk -F'"' '{print $4}')
echo "session id is: ${session_id}"

# authenticate the session
echo "authenticating the session"
authentication_resp=$(curl -s -k --url ${PID_SERVER}/api/sessions/this -X PUT --key ${PRIVKEY} --cert ${CERTIFICATE} -H "Content-Type:application/json" -H "Authorization: Handle clientCert="true", sessionId="${session_id}"")
echo "authenticating the session response is: ${authentication_resp}"

# create a handle
echo "creating several handles"
SUFFIX=$(openssl rand -base64 60 | tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1) #generates random alphanumeric string of length 32
URL="http://www.test.com" 
echo "Creating handle $PREFIX/$SUFFIX pointing to $URL"
create_resp=$(curl -s -k --url ${PID_SERVER}/api/handles/${PREFIX}/${SUFFIX} -X PUT -H 'Content-Type: application/json' -H 'Accept: application/json' -H "Authorization: Handle sessionId="${session_id}"" -d  '{"values": [{"index":1,"type":"URL","data":{"format":"string","value":${URL}}},{"index":100,"type":"HS_ADMIN","data":{"format":"admin","value":{"handle":"0.NA/${PREFIX}","index":200,"permissions":"011111110011"}}}]}')
echo "creating handle 1 response is: ${create_resp}"

# create a second handle
SUFFIX=$(openssl rand -base64 60 | tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1) #generates random alphanumeric string of length 32
URL="http://www.test.com" 
echo "Creating handle $PREFIX/$SUFFIX pointing to $URL" 
create_resp=$(curl -s -k --url ${PID_SERVER}/api/handles/${PREFIX}/${SUFFIX} -X PUT -H 'Content-Type: application/json' -H 'Accept: application/json' -H "Authorization: Handle sessionId="${session_id}"" -d  '{"values": [{"index":1,"type":"URL","data":{"format":"string","value":${URL}}},{"index":100,"type":"HS_ADMIN","data":{"format":"admin","value":{"handle":"0.NA/${PREFIX}","index":200,"permissions":"011111110011"}}}]}')
echo "creating handle 2 response is: ${create_resp}"

# create a third handle
SUFFIX=$(openssl rand -base64 60 | tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1) #generates random alphanumeric string of length 32
URL="http://www.test.com" 
echo "Creating handle $PREFIX/$SUFFIX pointing to $URL"
create_resp=$(curl -s -k --url ${PID_SERVER}/api/handles/${PREFIX}/${SUFFIX} -X PUT -H 'Content-Type: application/json' -H 'Accept: application/json' -H "Authorization: Handle sessionId="${session_id}"" -d  '{"values": [{"index":1,"type":"URL","data":{"format":"string","value":${URL}}},{"index":100,"type":"HS_ADMIN","data":{"format":"admin","value":{"handle":"0.NA/${PREFIX}","index":200,"permissions":"011111110011"}}}]}')
echo "creating handle 3 response is: ${create_resp}"

# verify session
echo "verifying the session"
verify_rep=$(curl -s -k --url ${PID_SERVER}/api/sessions/this -X GET -H "Content-Type:application/json" -H "Authorization: Handle sessionId="${session_id}"")
echo "verifying session response is: ${verify_rep}"

# delete session
echo "deleting the session"
delete_resp=$(curl -s -k --url ${PID_SERVER}/api/sessions/this -X DELETE -H "Content-Type:application/json" -H "Authorization: Handle sessionId="${session_id}"" --write-out %{http_code} --output /dev/null)
echo "successful session deletion http(s) response should be 204 and is: ${delete_resp}"

# verify session
echo "verifying the session"
verify_rep=$(curl -s -k --url ${PID_SERVER}/api/sessions/this -X GET -H "Content-Type:application/json" -H "Authorization: Handle sessionId="${session_id}"")
echo "verifying the session response should now be empty and is: ${verify_rep}"

List the available handles for a prefix

Assuming you have access to a prefix, it is possible to list the handles which are present for said prefix.

The following script retrieves all handles created for a given prefix. Remember to replace any fields with "<>" brackets with your own values.

Please do NOT list more than 10 000 handles at a time. Otherwise the handle server will be overflowed!

Do NOT forget the "\" in front of the "&" signs. Otherwise it will be done in the background and the pageSize and pagewill not be taken into account!

#!/bin/bash

#### modify following lines ####
# Path to server
PORT=<port>
PID_SERVER=https://epic-pid.storage.surfsara.nl:${PORT}/api/handles

# PID
PREFIX=<prefix>  

# Authentication  
# assuming <INDEX>:<PREFIX>/USER01 for authentication
INDEX=<index>
USER="USER01"
MY_PATH="<path to folder with key/certificate>" 
PRIVKEY=${MY_PATH}/${PREFIX}_${USER}_${INDEX}_privkey.pem
CERTIFICATE=${MY_PATH}/${PREFIX}_${USER}_${INDEX}_certificate_only.pem
#### end modify lines ####

curl -v -k --key $PRIVKEY --cert $CERTIFICATE -H "Content-Type: application/json" -H 'Authorization: Handle clientCert="true"' -X GET $PID_SERVER?prefix=$PREFIX\&pageSize=10000\&page=0
curl -v -k --key $PRIVKEY --cert $CERTIFICATE -H "Content-Type: application/json" -H 'Authorization: Handle clientCert="true"' -X GET $PID_SERVER?prefix=$PREFIX\&pageSize=10000\&page=1
...
curl -v -k --key $PRIVKEY --cert $CERTIFICATE -H "Content-Type: application/json" -H 'Authorization: Handle clientCert="true"' -X GET $PID_SERVER?prefix=$PREFIX\&pageSize=10000\&page=n

More examples

See the EUDAT Training GitHub repository for more examples.

Space shortcuts

Page tree

Setup

Setting up curl for Mac OS users

Sessions

Example bash scripts using curl

Create a handle

Show handle contents

Modify a handle

Create multiple handles in a session

List the available handles for a prefix

More examples

Table of contents

Space shortcuts

Page tree

PIDs: Handle API using bash

Setup

Setting up curl for Mac OS users

Sessions

Example bash scripts using curl

Create a handle

Show handle contents

Modify a handle

Create multiple handles in a session

List the available handles for a prefix

More examples

Table of contents