Contact person: Go lang Application for File Parsing and Bulk SQL Operations

Phone:Show

Email:Show

Location: Prague, Czech Republic

Budget: Recommended by industry experts

Time to start: As soon as possible

Project description:
"Looking for a Go (Golang) Developer

Hello,
We are looking for an experienced Go (Golang) developer (or team) to create a console application that will parse downloaded files and store their content in a MySQL database.
A key requirement is efficient database handling – the application must generate bulk SQL queries (INSERT and UPDATE) rather than processing individual records one by one.
The complete technical specification can be found below.

If cooperation goes well, we are open to long-term collaboration on future projects.
Our stack includes: React, React Native, PHP, Symfony, CodeIgniter, Qt.


What We Offer
- A prepared environment: Dockerfile, docker-compose, and database data
- Consultations and clarification of any unclear parts of the specification
- Reliable and timely payments
- Fair and open communication

Requirements
- Clean and readable code, with clear comments on complex sections – all in English
- Reasonable pricing while maintaining high quality
- Use of AI tools where it makes sense to speed up development
- Regular GIT pushes at least once a week (on an agreed day)
- Data security and confidentiality – no sharing of the work created for us
- Full transfer of all licenses and rights to the final product upon delivery

Please send us your final fixed price for the complete project as defined below. If you work with an hourly rate, please also include your final hourly rate.
When selecting a developer, we will consider your experience, references, communication quality, and overall price offer.

We look forward to your proposal and potential cooperation!



Complete Specification:

1) Application Startup
The application will be run with the following parameters (if any are missing, help will be displayed):
a) database connection credentials
b) path to the folder containing configuration files (mapping: input file → database)
c) path to the folder for temporary files

2) Logging
A log entry will be stored in the database at application startup (including start time).

3) Database Initialization
a) Allowed URL address – selected based on the oldest update or priority defined in the database
b) File format corresponding to that URL
c) All existing record IDs for the given URL

4) Mapping Loading
Based on the format from point 3b), the application loads mapping from the configuration file (defines relation: XML element → database column).

5) File Downloading
The XML file is downloaded from the URL defined in 3a).
(Not part of the current requirement, but keep in mind for future updates: the design should allow an extendable interface to support additional formats, e.g., JSON, etc.)

6) Partial File Processing
The file is processed in a loop, with a predefined number of XML root elements per iteration.

7) Data Preparation for Storage
Data from the current iteration is transformed into an internal structure:
a) divided based on the existence of the unique record ID (3c) into INSERT and UPDATE batches
b) mapped from XML elements to database columns according to the configuration from point 4)

8) Fetching Records for Update
After each iteration, all records marked for update are fetched from the database.

9) Bulk Update (UPDATE)
A bulk UPDATE query is generated for all records from point 8), updating only those rows and fields where changes occurred.
The query is then sent to the database.
If a record was previously deactivated, it should be reactivated in the database.

10) Bulk Insert (INSERT)
A bulk INSERT query is generated for new records and sent to the database.
The INSERT batch may contain more records than the UPDATE batch.

11) Continued Processing
The loop returns to point 7) and continues until the entire file is processed.

12) Deactivation of Outdated Records
After processing is complete, records that are not present in the source file (i.e., not part of INSERT or UPDATE) will be deactivated in the database.

13) Final Logging
The final log entry is written to the database, including the end time and the number of updated products." (client-provided description)