java - More efficient method for storing records from a file -

recently in discussion asked: have one flat file containing many records 5 million. need write java program can fetch records file , store in database via jdbc. efficient approach?

my suggestion create:

• one thread handle jdbc connection (additionally can make connection class singleton)
• another thread fetch records file & save in table.
• additionally when number of records saved in database, 100 commit first continue.

here stored procedures better or there other way?

sounds have right approach in mind.

the cost , time spent on network i/o , db operations going larger file io , parsing time spent on flat file. there might small performance gain in having separate thread read file , prepare record db, wouldn't worth added complexity, , maybe not worth time jvm spend on thread management. i'd recommend:

• 1 thread read file , submit db updates.
• as bitfiddler says, use preparedstatements batch updates (preparedstatement.addbatch()) every record, , submit batch (preparedstatement.executebatch()) every "n" records. may want prototype see ideal value of "n" is, 100 place start.

i don't recommend stored procedures. won't if you're doing straight inserts.