SFTP - Getting started in Peliqan

You can handle files on an SFTP server in Peliqan using the SFTP connector. Add an SFTP connection under “Connections” first (host, port, username, password etc.). Next, write a Python script in Peliqan to e.g. import data from CSV or JSON files. In your Python script you can read the data and insert into a table in your data warehouse.

Example script to read a JSON file from an SFTP server:

dw = pq.dbconnect(pq.DW_NAME)     # your data warehouse
sftp = pq.sftpconnect("SFTP")

files = sftp.dir("/")
for file in files["detail"]:
	if file["type"] == "file": # type can be "file" or "folder"
		jsonStr = sftp.read_file(file["name"])  # assuming one object in file
		record = json.loads(jsonStr)
		dw.insert("my_db", "my_schema", "my_table", record)