Flat File Importer
Status: Prioritization Needed
User Story
As a DataShop administrator, I want to redesign Dataset Import Tool to load the dataset into database by processing it column by column so that I can speed up the import process.
Notes/Comments
- Current Dataset Import Tool processes teh dataset row by row and uses Hibernate layer, which takes a long time to import a dataset.
- The import sometime has failed for several large datasets.
- We'd like to rewrite this import tool to allow column by column process and avoid Hibernate layer to make the import faster.
- The goal is to process 1 million rows per minute on import only. Shanwen10:09, 12 October 2010 (EDT)
See complete DataShop Feature Wish List.