Massive DataFrame Display Optimization on Streamlit
Budget: $30 – $250 USD
Project Objective: Efficiently Display a Massive DataFrame in Streamlit
Description:
We have a large DataFrame (~120,000 rows × ~1,000 columns) that needs to be displayed entirely on a Streamlit-based website. The key challenge is performance—we need a solution that rapidly loads and smoothly renders the entire DataFrame in the user's browser, despite having limited server resources. The psych_table.csv is the DataFrame that should be displayed.
Requirements:
• Display the entire DataFrame (~120K rows, ~1K columns) simultaneously—pagination is NOT allowed.
• Optimize both server-side and client-side performance.
• Employ JavaScript or other programming languages if needed, provided the implementation can be seamlessly embedded into existing Streamlit code.
• Ensure compatibility with Streamlit and maintain a smooth user experience without significant loading times.
• The DataFrame includes a column named 'link'. Each link should be displayed as a clickable hyperlink with an emoji ? as the display text.
• Column width customization is required:
o Some columns (e.g., 'Item DB', 'Variable ID') should have a small width.
o Some columns (e.g., 'Item Text') should have a wider width.
• Provide the download option to download the whole dataframe as csv file.
Deliverables:
• Python/Streamlit code or components implementing the solution, including any custom column configuration.
• Any necessary JavaScript or other language code with clear instructions for integration into Streamlit.
• Documentation outlining the implementation and instructions for integration.
• Brief summary explaining chosen optimization techniques.
Please apply only if you have experience handling large-scale data visualization or optimization tasks within Streamlit.
File example: https://www.dropbox.com/scl/fi/kxx23f69j4gq1wue8p8ko/psych_table.csv?rlkey=cagg2n8u17czyqous6nvnaaid&e=1&dl=0
Description:
We have a large DataFrame (~120,000 rows × ~1,000 columns) that needs to be displayed entirely on a Streamlit-based website. The key challenge is performance—we need a solution that rapidly loads and smoothly renders the entire DataFrame in the user's browser, despite having limited server resources. The psych_table.csv is the DataFrame that should be displayed.
Requirements:
• Display the entire DataFrame (~120K rows, ~1K columns) simultaneously—pagination is NOT allowed.
• Optimize both server-side and client-side performance.
• Employ JavaScript or other programming languages if needed, provided the implementation can be seamlessly embedded into existing Streamlit code.
• Ensure compatibility with Streamlit and maintain a smooth user experience without significant loading times.
• The DataFrame includes a column named 'link'. Each link should be displayed as a clickable hyperlink with an emoji ? as the display text.
• Column width customization is required:
o Some columns (e.g., 'Item DB', 'Variable ID') should have a small width.
o Some columns (e.g., 'Item Text') should have a wider width.
• Provide the download option to download the whole dataframe as csv file.
Deliverables:
• Python/Streamlit code or components implementing the solution, including any custom column configuration.
• Any necessary JavaScript or other language code with clear instructions for integration into Streamlit.
• Documentation outlining the implementation and instructions for integration.
• Brief summary explaining chosen optimization techniques.
Please apply only if you have experience handling large-scale data visualization or optimization tasks within Streamlit.
File example: https://www.dropbox.com/scl/fi/kxx23f69j4gq1wue8p8ko/psych_table.csv?rlkey=cagg2n8u17czyqous6nvnaaid&e=1&dl=0