Uploading and Downloading Files

Uploading and Downloading Files#

Uploading a File#

Usage notes:

  • Currently, only parquet files are supported. Snappy compression is supported.

  • You can only upload one file at a time via the API. You must make multiple requests to upload multiple files. The Python SDK supports uploading multiple files at once.

  • All files must have the same schema. Files with schemas that do not match will be rejected.

  • Names must be unique. If you upload a file with a name that already exists in the stage, it will be rejected.

  • When you upload to a stage, the ordering will be preserved. If you upload multiple files to a stage, they will be added to the stage in the order you provide them.

POST https://api.materialized.dev/upload-to-stage#

Upload a file to a stage.

Request Headers:
Request JSON Object:

  • stage_id (str) – The ID of the stage to upload the file to.

  • file (str) – The file to upload.

Returns:

A JSON object containing the file ID.

Example

import requests
import json

url = "https://api.materialized.dev/upload-to-stage"
files = {
    "file": ("test.parquet", open("test.parquet", "rb"), "application/octet-stream")
}

payload = {
    "stage_id": stage_id,
}

headers = {
    "Authorization": "Key YOUR_API_KEY"
}

response = requests.post(url, headers=headers, data=payload, files=files)
print(response.json())

Downloading a File#

Usage notes:

  • You can only download one file at a time via the API. You must make multiple requests to download multiple files. The Python SDK supports downloading multiple files at once.

  • The file will be returned as bytes. You will need to decode the bytes and save them to a file.

  • The stage stores files in the order they were uploaded. You can view the ordering of files using the list-stage-files endpoint.

POST https://api.materialized.dev/download-from-stage#

Download a file from a stage.

Request Headers:
Request JSON Object:

  • stage_id (str) – The ID of the stage to download the file from.

  • file_name (str) – The name of the file to download.

Returns:

If successful, the file, as bytes. Otherwise, a JSON object containing an error message.

Example

import requests
import json
import base64

url = "https://api.materialized.dev/download-from-stage"
payload = {
    "stage_id": stage_id,
    "file_name": file_name
}
headers = {
    "Authorization": "Key YOUR_API_KEY"
}

response = requests.post(url, headers=headers, data=payload)

if response.status_code != 200:
    print(response.json()['message'])
else:
    with open("downloaded_file.parquet", "wb") as f:
        f.write(response.content)
Contents