SFTP

You can handle files on an SFTP server in Peliqan using the SFTP connector. Add an SFTP connection under “Connections” first (host, port, username, password etc.). Next, write a Python script in Peliqan to e.g. import data from CSV or JSON files. In your Python script you can read the data and insert into a table in your data warehouse.

Example script to read a JSON file from an SFTP server:

dw = pq.dbconnect("dw_123")     # your data warehouse
sftp = pq.sftpconnect("SFTP")

files = sftp.dir("/")
for file in files["detail"]:
	if file["type"] == "file": # type can be "file" or "folder"
		jsonStr = sftp.read_file(file["name"])  # assuming one object in file
		record = json.loads(jsonStr)
		dw.insert("my_db", "my_schema", "my_table", record)

You can also deploy an SFTP server from the Peliqan market place !