I made a script and I am doing a few things in it. I am uploading 10,000 documents and a JSON files to IPFS through Infura, and I am writing data to an Excel speadsheet. The Excel part works fine, I am only mentioning that in case it matters. I start the loop, and pretty early on I get raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ( Connection aborted. , ConnectionResetError(54, Connection reset by peer ))
I have seen similar posts, including this, but I read somewhere else that if you strategically put sleep(0.01)
around your code, it will help with this. While I have seen improvements, this issue still persists. I am not sure if I placed them right or if I can do something else. This is the short version of my script:
import time
import requests
from xlwt import Workbook
import json
### IPFS Integration
for i in range(1, 10000 + 1):
# Create image file object
image_file = {
file : open( images/ + str(i) + .png , rb )
}
time.sleep(0.01)
### Upload image to IPFS
image_response = requests.post(endpoint + /api/v0/add , files=image_file, auth=(project_id, project_secret))
time.sleep(0.01)
### Change metadata image url
# Create raw JSON file
json_file = open( newjson/ + str(i) + .json , r+ )
raw_json = json.load(json_file)
json_file.close()
# Change image URL
raw_json[ image ] = image_url
time.sleep(0.01)
# Save raw JSON
json_file = open( newjson/ + str(i) + .json , w )
json.dump(raw_json, json_file)
json_file.close()
# Create metadata file object
meta_file = {
file : open( newjson/ + str(i) + .json , rb )
}
time.sleep(0.01)
### Upload metadata to IPFS
meta_response = requests.post(endpoint + /api/v0/add , files=meta_file, auth=(project_id, project_secret))
params = {
arg : meta_cid
}
time.sleep(0.01)
meta_info_response = requests.post(endpoint + /api/v0/cat , params=params, auth=(project_id, project_secret))
meta_sheet.write(i, json_col, meta_info_response.text)
I got rid of a lot of variables for privacy purposes, but how does this look? I copied and pasted a lot so I may have forgotten some chunks. But am I putting the sleeps in the right spot? Usually I will get an error from round 9 till 25. Please let me know if I can get some help, thank you!