Need guidance with python
Now, I have script which imports data into SQLite3 database from .json file. Current code:
# Auto-discover columns
columns = []
column = []
for data in json_data:
column = list(data.keys())
for col in column:
if col not in columns:
columns.append(col)
# Auto-bind values to discovered columns
value = []
values = []
for data in json_data:
for i in columns:
value.append(str(dict(data).get(i)))
values.append(list(value))
value.clear()
insert_query = 'insert into logs values (null,:date,:server,:ip,:mta,:country,:env_from,:from,:subject,:spam_type,:mqid,:sender_domain,:auth_id,:verdict,:class,:score,:raw,:pure_log)'
c = db.cursor()
c.executemany(insert_query, values)
db.commit()
There is 2 data points which need pre-processing before insertion: 'env_from' and 'subject'. How to do that? I guess I need to modify '# Auto-bind values to discovered columns' part of code. But whatever I do - it just rejects with error. Specifically I want to decode strings in utf8 mime (email).
Thanked by (1)Logano
Comments
I wish I could help, but I haven't taken or read any of the 50-plus Python courses and books that are in my backlog.
Thank you for good intention it is appreciated.
Ha, found a solution:
Not related to solving a problem, but:
I would recommend using FastAPI with sqlalchemy to achieve a similar effect, which I find a bit more lightweight.
My virtualized amusement park: https://github.com/oneclickvirt