3-Tier Data Storage System with Encryption and Decentralized immutability
Budget: €250 – €750 EUR
Task Description: Implement 3-Tier Data Storage framework with Encryption and Decentralized Backup
Project Overview:
We are building a 3-tier storage system that involves handling data (binary, excel, database, etc.) and metadata (project data, clusters, data points, relationships, etc.) with a local, remote, and decentralized storage architecture. The system will ensure data encryption and immutable storage across three different tiers:
1. Tier 1: Local Storage (Client Machine)
2. Tier 2: Remote Storage (Institution/Company Server)
3. Tier 3: Decentralized Storage (Blockchain/Distributed)
The project involves developing the entire workflow, from the importing of data to its encryption and synchronization across these tiers. Data and metadata must be versioned and synced across all layers to ensure collaboration and redundancy.
Key Requirements:
1. Tier 1: Local Storage (Client Machine)
• Local Data Encryption: Raw data (such as binary files, excel sheets, database, etc.) is uploaded to the system and must be encrypted using the owner’s public key i.e. using Yubikey before storing it locally.
• Local Metadata Repo: A local Git repository should store metadata (e.g., clusters, data points, relationships) related to the data.
• Client-side Application: The application should allow users to manage their local data and metadata and perform encryption operations seamlessly.
2. Tier 2: Remote Storage (Institution/Company)
• Data Syncing: The encrypted data from Tier 1 should be synchronized with a remote server. The remote server should store encrypted data in its own storage.
• Remote Metadata Sync: The project metadata (Git repository) should also be synchronized with the remote server, ensuring version control of the metadata.
• Collaboration Features (Out of scope): Enable multiple users to collaborate on the project, with access to both the encrypted data and the project metadata, allowing seamless synchronization.
3. Tier 3: Decentralized Storage (Blockchain/Distributed)
• Decentralized Data storage: The encrypted data should be moved to a decentralized storage solution (IPFS) for long-term storage and backup.
• Blockchain Integration: Store project metadata immutably on a blockchain EVM platform (e.g., EVM Test net) to guarantee the tamper-proof nature of project metadata and data relationships.
• Specific version immutability: Ensure that metadata and data relationships specific version will be permanently stored on the blockchain.
Functionality to Implement:
1. Data Upload and Encryption:
• Implement an encryption service that encrypts raw data using the owner’s public key (Yubikey) before uploading it to Tier 1 (local storage).
2. Synchronization Logic:
• Sync encrypted data and selected project metadata versions from Tier 1 to Tier 2 (remote server), enabling versioned storage and version control.
• Sync project data and metadata specific finished version from Tier 2 to Tier 3 (blockchain and decentralized storage) to maintain an immutable state of data and metadata.
3. Data and Metadata Management:
• Users should be able to view and manage Projects, Data clusters, data points, and relationships from within the application.
• Implement Git-based versioning for project metadata across Tier 1 and select versions to be synced to Tier 2 And select the finished project version to be synced to Tier 3.
• Ensure that decentralized storage on IPFS and blockchain can retrieve data and metadata reliably.
4. Security and Access Control:
• Implement access control to ensure only authorized users can access, view, and modify the data and project metadata (basic account access management).
• Ensure encrypted data to be the only data sent out from Tier 1 and that only the owner can decrypt the data.
Tech Stack Requirements:
• Frontend: Any suitable frontend stack for client-side interactions (e.g., React, Angular, Vue.js) web app preferred.
• Backend: Node.js, Python, or any backend that supports file operations, encryption, and API-based communication with external services.
• Encryption: Use RSA encryption for data encryption (owner’s public key encryption) using a Yubikey hardware.
• Git: Git repository management for project metadata versioning.
• Blockchain: EVM test net for storing metadata immutably on the blockchain.
• Decentralized Storage: IPFS for storing data in a decentralized way.
Deliverables:
• A working prototype or POC of the 3-tier storage system, with local, remote, and decentralized tiers implemented.
• Complete implementation of data encryption, syncing, and storage across the tiers.
• A simple frontend interface to manage data and metadata, view encrypted files, and perform syncing operations.
• Full documentation on the architecture, how to set up the system, and how to use the application.
• Source code of all components + comments.
Timeline:
• Estimated time for completion: 4-6 weeks (Flexible based on your experience and progress).
Skills Required:
• Knowledge of encryption algorithms (e.g., RSA, AES).
• Experience with decentralized storage systems like IPFS .
• Git expertise for managing version control of project metadata.
• Experience in blockchain technologies (EVM, smart contracts).
• Proficiency in frontend development (React, Vue.js, Angular) for building user interfaces.
• Proficiency in backend technologies (Node.js, Python) for implementing the syncing and encryption services.
How to Apply:
1. Please provide a detailed proposal, including your experience with encryption, decentralized storage systems, and blockchain technologies.
2. Share examples of similar projects you’ve worked on, especially those involving data encryption and decentralized storage.
3. Include an estimated timeline and a cost breakdown for each phase of the project.
Project Overview:
We are building a 3-tier storage system that involves handling data (binary, excel, database, etc.) and metadata (project data, clusters, data points, relationships, etc.) with a local, remote, and decentralized storage architecture. The system will ensure data encryption and immutable storage across three different tiers:
1. Tier 1: Local Storage (Client Machine)
2. Tier 2: Remote Storage (Institution/Company Server)
3. Tier 3: Decentralized Storage (Blockchain/Distributed)
The project involves developing the entire workflow, from the importing of data to its encryption and synchronization across these tiers. Data and metadata must be versioned and synced across all layers to ensure collaboration and redundancy.
Key Requirements:
1. Tier 1: Local Storage (Client Machine)
• Local Data Encryption: Raw data (such as binary files, excel sheets, database, etc.) is uploaded to the system and must be encrypted using the owner’s public key i.e. using Yubikey before storing it locally.
• Local Metadata Repo: A local Git repository should store metadata (e.g., clusters, data points, relationships) related to the data.
• Client-side Application: The application should allow users to manage their local data and metadata and perform encryption operations seamlessly.
2. Tier 2: Remote Storage (Institution/Company)
• Data Syncing: The encrypted data from Tier 1 should be synchronized with a remote server. The remote server should store encrypted data in its own storage.
• Remote Metadata Sync: The project metadata (Git repository) should also be synchronized with the remote server, ensuring version control of the metadata.
• Collaboration Features (Out of scope): Enable multiple users to collaborate on the project, with access to both the encrypted data and the project metadata, allowing seamless synchronization.
3. Tier 3: Decentralized Storage (Blockchain/Distributed)
• Decentralized Data storage: The encrypted data should be moved to a decentralized storage solution (IPFS) for long-term storage and backup.
• Blockchain Integration: Store project metadata immutably on a blockchain EVM platform (e.g., EVM Test net) to guarantee the tamper-proof nature of project metadata and data relationships.
• Specific version immutability: Ensure that metadata and data relationships specific version will be permanently stored on the blockchain.
Functionality to Implement:
1. Data Upload and Encryption:
• Implement an encryption service that encrypts raw data using the owner’s public key (Yubikey) before uploading it to Tier 1 (local storage).
2. Synchronization Logic:
• Sync encrypted data and selected project metadata versions from Tier 1 to Tier 2 (remote server), enabling versioned storage and version control.
• Sync project data and metadata specific finished version from Tier 2 to Tier 3 (blockchain and decentralized storage) to maintain an immutable state of data and metadata.
3. Data and Metadata Management:
• Users should be able to view and manage Projects, Data clusters, data points, and relationships from within the application.
• Implement Git-based versioning for project metadata across Tier 1 and select versions to be synced to Tier 2 And select the finished project version to be synced to Tier 3.
• Ensure that decentralized storage on IPFS and blockchain can retrieve data and metadata reliably.
4. Security and Access Control:
• Implement access control to ensure only authorized users can access, view, and modify the data and project metadata (basic account access management).
• Ensure encrypted data to be the only data sent out from Tier 1 and that only the owner can decrypt the data.
Tech Stack Requirements:
• Frontend: Any suitable frontend stack for client-side interactions (e.g., React, Angular, Vue.js) web app preferred.
• Backend: Node.js, Python, or any backend that supports file operations, encryption, and API-based communication with external services.
• Encryption: Use RSA encryption for data encryption (owner’s public key encryption) using a Yubikey hardware.
• Git: Git repository management for project metadata versioning.
• Blockchain: EVM test net for storing metadata immutably on the blockchain.
• Decentralized Storage: IPFS for storing data in a decentralized way.
Deliverables:
• A working prototype or POC of the 3-tier storage system, with local, remote, and decentralized tiers implemented.
• Complete implementation of data encryption, syncing, and storage across the tiers.
• A simple frontend interface to manage data and metadata, view encrypted files, and perform syncing operations.
• Full documentation on the architecture, how to set up the system, and how to use the application.
• Source code of all components + comments.
Timeline:
• Estimated time for completion: 4-6 weeks (Flexible based on your experience and progress).
Skills Required:
• Knowledge of encryption algorithms (e.g., RSA, AES).
• Experience with decentralized storage systems like IPFS .
• Git expertise for managing version control of project metadata.
• Experience in blockchain technologies (EVM, smart contracts).
• Proficiency in frontend development (React, Vue.js, Angular) for building user interfaces.
• Proficiency in backend technologies (Node.js, Python) for implementing the syncing and encryption services.
How to Apply:
1. Please provide a detailed proposal, including your experience with encryption, decentralized storage systems, and blockchain technologies.
2. Share examples of similar projects you’ve worked on, especially those involving data encryption and decentralized storage.
3. Include an estimated timeline and a cost breakdown for each phase of the project.