SPOT provides a REST API that can be used in variety of languages (javascript, BASH, python, JAVA etc). The output is typically JSON formatted data.
To use the SPOT REST API, you must first authenticate to NEWT using your NIM credentials. NEWT will soon support Multi-Factor Authentication (MFA) and so will SPOT's REST API. In order to simplify the API and ensure security in the SPOT back-end, the authentication process requires two steps by users of the SPOT API:
newt_sessionid
.
newt_sessionid
received from NEWT.
Once authenticated to the SPOT API,
the newt_sessionid
is used for future calls to
the API. For curl
commands a cookies file is
used, whereas in Python the session object will take care of
it.
Examples of how to do this in Bash (using curl) and Python are shown below.
For long running sessions, if either the NEWT or API session lifetimes expires, then re-authenticating will be necessary.
$ curl -sS -X POST -d "username=<your_username>&password=<your_nim_password><nim_mfa_otp>" https://newt.nersc.gov/newt/auth | jq .
{ "username": "[your_username]", "session_lifetime": 43199, "auth": true, "newt_sessionid": "43cf49b59ab3608cc2e7ff380c7b02a5" }Confirm that values are correct (auth is true, username is yours, lifetime is > 0) then use
newt_sessionid
as password for authenticating to the SPOT API:$ curl -c als_cookies.txt -sS -X POST -d "username=<your_username>&password=43cf49b59ab3608cc2e7ff380c7b02a5" https://portal-auth.nersc.gov/als/auth | jq .
{ "username": "[your_username]", "session_lifetime": 566317, "auth": true }The output of this command is just informative, the
als_cookies.txt
file is what to
use for the real calls to the SPOT API:
$ curl -b als_cookies.txt -sS -X GET "https://portal-auth.nersc.gov/als/hdf/search?limitnum=1&end_station=bl832" | jq .
[ { "fs": { "size": 4655890616, "derivatives": [ { "dstage": "fast-tomopy", ...
# Credentials: $ user="ksb" $ otp="1234" # If MFA enabled for your NIM account $ pw="Ch@ngeM3" # Auth to NEWT: $ newt_sessionid=$(curl -sS -X POST -d "username=${user}&password=${pw}${otp}" https://newt.nersc.gov/newt/auth | jq -r .newt_sessionid) # Auth to SPOT: $ curl -c api_cookies.txt -sS -X POST -d "username=${user}&password=${newt_sessionid}" https://portal-auth.nersc.gov/als/auth > /dev/null # Make REST call: $ curl -b als_cookies.txt -sS -X GET "https://portal-auth.nersc.gov/als/hdf/search?limitnum=1&end_station=bl832" | jq .
$ curl -c als_cookies.txt -sS -X GET "https://portal-auth.nersc.gov/als/hdf/search?skipnum=0&limitnum=10&sortterm=fs.stage_date&sorttype=desc&search=end_station=bl832"
$ curl -c als_cookies.txt -sS -X GET "https://portal-auth.nersc.gov/als/hdf/dataset?dataset=20130713_185717_Chilarchaea_quellon_F_9053427_IKI_"
$ curl -c als_cookies.txt -sS -X GET "https://portal-auth.nersc.gov/als/hdf/attributes/als/bl832/hmwood/20130713_185717_Chilarchaea_quellon_F_9053427_IKI_/raw/20130713_185717_Chilarchaea_quellon_F_9053427_IKI_.h5?group=/"
$ curl -c als_cookies.txt -sS -X GET "https://portal-auth.nersc.gov/als/hdf/listimages/als/bl832/hmwood/20130713_185717_Chilarchaea_quellon_F_9053427_IKI_/raw/20130713_185717_Chilarchaea_quellon_F_9053427_IKI_.h5"
$ curl -c als_cookies.txt -sS -X GET "https://portal-auth.nersc.gov/als/hdf/stageifneeded/als/bl832/hmwood/20130713_185717_Chilarchaea_quellon_F_9053427_IKI_/raw/20130713_185717_Chilarchaea_quellon_F_9053427_IKI_.h5"
$ curl -c als_cookies.txt -sS -X GET "https://portal-auth.nersc.gov/als/hdf/download/als/bl832/hmwood/20130713_185717_Chilarchaea_quellon_F_9053427_IKI_/raw/20130713_185717_Chilarchaea_quellon_F_9053427_IKI_.h" > file.h5
$ curl -c als_cookies.txt -sS -X GET "https://portal-auth.nersc.gov/als/hdf/rawdata/als/bl832/hmwood/20130713_185717_Chilarchaea_quellon_F_9053427_IKI_/norm/20130713_185717_Chilarchaea_quellon_F_9053427_IKI_-norm-20130714_192637.h5?group=/20130713_185717_Chilarchaea_quellon_F_9053427_IKI_/20130713_185717_Chilarchaea_quellon_F_9053427_IKI__0000_0640.tif"
$ curl -c als_cookies.txt -sS -X GET "https://portal-auth.nersc.gov/als/hdf/image/als/bl832/hmwood/20130713_185717_Chilarchaea_quellon_F_9053427_IKI_/norm/20130713_185717_Chilarchaea_quellon_F_9053427_IKI_-norm-20130714_192637.h5?group=/20130713_185717_Chilarchaea_quellon_F_9053427_IKI_/20130713_185717_Chilarchaea_quellon_F_9053427_IKI__0000_0640.tif"
$ curl -c als_cookies.txt -sS -X GET "https://portal-auth.nersc.gov/als/hdf/tomopyjob?dataset=20130713_185717_Chilarchaea_quellon_F_9053427_IKI_"
import requests
newt_session = requests.Session() newt_response = newt_session.post("https://newt.nersc.gov/newt/auth", {"username": "ksb", "password": "<NIM password>" + "<MFA OTP>"}) newt_response.json()
{'auth': True, 'newt_sessionid': 'bda90e628b81752wx27dca7b0a21e8z0', 'session_lifetime': 43199, 'username': 'ksb'}
newt_sessionid = newt_response.json()['newt_sessionid']
api_session = requests.Session() api_response = api_session.post("https://portal-auth.nersc.gov/als/auth", {"username": "ksb", "password": newt_sessionid}) api_response.json()
{"username": "spotdev", "session_lifetime": 43200, "auth": True}
api_response = api_session.get("https://portal-auth.nersc.gov/als/hdf/search?skipnum=0&limitnum=2&sortterm=fs.stage_date&sorttype=desc&search=end_station=bl832")Look at results:
api_response.json()