Đang Thực Hiện

Golang, postfix, email parsing

You will be writing 2 small command-line programs in Golang. These automate a workflow where images get emailed to a server, and the images get extracted and put into a database.

The architecture:

- There is a config file which gives details of which program to run, database connectivity, paths for s3 buckets and Amazon credentials

- An environment variable MICONF points to the config file (usually /etc/[login to view URL])

- A postgresql database (an extract from the schema is shown below)

- Various s3 buckets

---------------------------------------------

Program 1: /usr/local/bin/email-ingest

This program is run from a .forward rule in postfix. i.e. There is an account (ingester) and a file /home/ingester/.forward that contains the line

|/usr/local/bin/email-ingest

So emails to ingester@[login to view URL] end up as the stdin for /usr/local/bin/email-ingest.

What this program does:

- Creates a new random uuid for this message ($msgid)

- Looks up the config file for S3_RAW_EMAIL_BUCKET

- Saves stdin to s3://$S3_RAW_EMAIL_BUCKET/$msgid

- Creates a record in raw_emails_received

- Looks up the config file for EXEC_EMAIL_PARSE, which is likely to be /usr/local/bin/email-parse,

and executes it with -m $msgid

- Triggers a postgresql NOTIFY new_email, $msgid

-----------------------------------------------

Program 2: /usr/local/bin/email-parse -m $msgid

What it does:

- Looks up raw_emails_received for the record where message_id = $msgid, and locks that row in the table

- Reads the s3 file mentioned there and extracts out the From: To: Subject: and Date: fields . These are stored in the email_headers_received table.

- If the username part (excluding the @ and the domain) from To: field matches short_key in the clinics table, then matched_to_clinic = true, otherwise it is false. If it is true, then a record is added to clinic_emails.

- Any JPEG images which are attachments to the message are extracted out. These will be stored in the bucket defined by S3_RAW_IMAGE_BUCKET from the config file. For each image:

- A new uuid is created for the image ($imageid)

- A new record is created in images_received, including height and width

- The data of the image is put into s3://$S3_RAW_IMAGE_BUCKET$imageid

- Raw_emails_received gets updated so that processed = true, and then unlocks the record

- Triggers a postgresql NOTIFY images_ready, $msgid

create table clinics (

clinic_id uuid primary key,

short_key varchar not null unique,

clinician_sending_email varchar unique,

billing_email varchar,

imagery_destination_email not null varchar,

active boolean not null default true

);

create table raw_emails_received (

message_id uuid primary key,

s3_path varchar not null unique,

received_timestamp timestamp default timenow(),

processed boolean default false

);

create table email_headers_received (

message_id uuid references raw_emails_received,

from_field varchar not null,

to_field varchar not null,

subject varchar not null,

date_sent varchar not null,

matched_to_clinic boolean not null,

number_of_images int not null constraint number_of_images >= 0,

when_extracted timestamp default timenow()

);

create table clinic_emails_received (

message_id uuid references email_headers_received,

clinic_id uuid references clinics

);

create table images_received (

image_id uuid primary key,

source_email uuid reference raw_emails_received,

received_filename varchar, -- note that this comes from the user so can't be trusted or used for much

s3_path varchar not null,

height int check (height > 0),

width int check (width > 0),

extract_timestamp timestamp not null default timenow(),

);

Kĩ năng: Linux, Golang

Xem nhiều hơn: golang send email with attachment, golang smtp sendmail bcc, gomail golang, golang receive email, golang email api, golang read outlook email, golang send email gmail, golang email verification, free email parsing http post, email parsing php script, pop3 email parsing, email parsing subject, email parsing script, debian postfix email server, email parsing, email parsing sql, php email parsing application, email parsing application, sports email parsing, email parsing post server

Về Bên Thuê:
( 7 nhận xét ) Beecroft, Australia

ID dự án: #22252169

Được trao cho:

dharmjitsingh

Hi there, I have experience in golang and devops technologies.i have developed both restful and grpc microservices using standard library and external frameworks. for command line application, I have experience in cobr Thêm

$244 AUD trong 20 ngày
(0 Đánh Giá)
0.0

2 freelancer đang chào giá trung bình $222 cho công việc này

gopikishore

Hi Sir, I’m having hands on experience in back end projects by using golang. As an Golang developer, i’m having rich experience in using microservice, websockets with back end development. I have a good experience i Thêm

$200 AUD trong 9 ngày
(0 Nhận xét)
0.0