Two Autonomous Network Programs Written in Rust Lang
Budget: $100 – $2,500 USD
This is a new project to write two relatively simple programs in Rust. The goal of these
programs is to simulate functions that are carried out in FPGAs for the storage and retrieval of
data records. All functions are done in binary, that is to say, that the records are an array of
bytes and are not broken down into higher logical objects or structures. A GitHub repository will
be provided for this project. You will be expected to give short daily updates via the Freelancer
chat. Your proposal should include the size of your team, estimated time needed to complete
this project, references to past Rust Lang-based projects which used TCP/IP networking, Byte
Operations, and NVMe Block Storage which demonstrate an ability to complete this project.
The first program, referred to as RMM, will receive payloads of one or more records as TCP/IP
packets. Each packet will have a header that includes the batch that the records in that packet
belong to. Batches are sequentially delivered. This program will need to store these records in
4KB blocks directly to NVMe drives dedicated to this application without any filesystem, each
LBA should only hold records for a single batch. (Ubuntu 20.04 OS). The program will also need
to keep an index of the first block address (LBA) used for each batch of records. This array
should be stored or copped into nonvolatile memory in any format that the programer finds
appropriate. When requested, through a command network channel, this program will need to
in a separate process and using the index array, locate the first and last block (LBA) used for the
indicated record batches. Using a provided bitmask identify records in the batch matching the
mask and send them to the second program as a TCP/IP Payload. As scanning of each batch
is completed the program will need to notify the second program.
The second program, referred to as IQS, will receive payloads of records from one or more
RMM program instances. Using information provided to it externally via a command network
channel it will need to group records by specific byte sequences (for example all records having
the same byte sequence for bytes 4 through 7) and provided aggregates such as count of
matching records, or the min, max, avg, or sum of bytes 16 through 19 where every byte group
can be treated as an unsigned integer for each batch. When told by a sending RMM that it has
sent all records for a batch the running aggregates should be sent to a final target provided in
the initial command.
Flow charts and a non completely debugged Python model is available for you to work from.
The base system available for this project is a bare metal (Not Cloud) Ryzen 5950X with 128GB
of RAM and four Sabrent 1TB Rocket 4 Plus NVMe SSDs for data storage. OS and program
will run from a different m.2 SSD.
TimeLine: Two Weeks
programs is to simulate functions that are carried out in FPGAs for the storage and retrieval of
data records. All functions are done in binary, that is to say, that the records are an array of
bytes and are not broken down into higher logical objects or structures. A GitHub repository will
be provided for this project. You will be expected to give short daily updates via the Freelancer
chat. Your proposal should include the size of your team, estimated time needed to complete
this project, references to past Rust Lang-based projects which used TCP/IP networking, Byte
Operations, and NVMe Block Storage which demonstrate an ability to complete this project.
The first program, referred to as RMM, will receive payloads of one or more records as TCP/IP
packets. Each packet will have a header that includes the batch that the records in that packet
belong to. Batches are sequentially delivered. This program will need to store these records in
4KB blocks directly to NVMe drives dedicated to this application without any filesystem, each
LBA should only hold records for a single batch. (Ubuntu 20.04 OS). The program will also need
to keep an index of the first block address (LBA) used for each batch of records. This array
should be stored or copped into nonvolatile memory in any format that the programer finds
appropriate. When requested, through a command network channel, this program will need to
in a separate process and using the index array, locate the first and last block (LBA) used for the
indicated record batches. Using a provided bitmask identify records in the batch matching the
mask and send them to the second program as a TCP/IP Payload. As scanning of each batch
is completed the program will need to notify the second program.
The second program, referred to as IQS, will receive payloads of records from one or more
RMM program instances. Using information provided to it externally via a command network
channel it will need to group records by specific byte sequences (for example all records having
the same byte sequence for bytes 4 through 7) and provided aggregates such as count of
matching records, or the min, max, avg, or sum of bytes 16 through 19 where every byte group
can be treated as an unsigned integer for each batch. When told by a sending RMM that it has
sent all records for a batch the running aggregates should be sent to a final target provided in
the initial command.
Flow charts and a non completely debugged Python model is available for you to work from.
The base system available for this project is a bare metal (Not Cloud) Ryzen 5950X with 128GB
of RAM and four Sabrent 1TB Rocket 4 Plus NVMe SSDs for data storage. OS and program
will run from a different m.2 SSD.
TimeLine: Two Weeks