Flat File Parsing into SQL Server

Flat File Parsing into SQL Server

Page 1 of 1

1 Replies - 9124 Views - Last Post: 08 February 2006 - 08:49 AM Rate Topic: -----

#1 neural  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 12
  • Joined: 28-December 04

Flat File Parsing into SQL Server

Posted 08 February 2006 - 04:23 AM

Hi,
I was wondering if anybody knew of any other ways of efficiently parsing a flat file into SQL Server 2000 using C#. The flat files are tab delimited. And the general file size is around 1 GB so this code has to be ultra efficient. Its a colllection of few flat files which willl be mapped to a relational structure in the database.

Following are the options I am looking at:
1. Using Datasets to read the data and execute one stored procedure per row. I am not finding this efficient at all obviously.

2. Reading the flat file, rewriting it so as to map it exactly to a table and then using Interop to invoke Sql BCP utility. This approach looks good but I would like to explore a bit more and see if I can avoid using an interop.

Please only give me approaches you think can be explored. I am not after code here yet.

Thanks.

Regards
Neural

Is This A Good Question/Topic? 0
  • +

Replies To: Flat File Parsing into SQL Server

#2 Amadeus  Icon User is offline

  • g+ + -o drink whiskey.cpp
  • member icon

Reputation: 248
  • View blog
  • Posts: 13,506
  • Joined: 12-July 02

Re: Flat File Parsing into SQL Server

Posted 08 February 2006 - 08:49 AM

Is this a task that will have to be performed on arepetitive basis? If not, I would suggest you simply import the files into tables that you've created.

Although it sounds simplistic, one of the best approaches it to simply read from the text file as a string, split the data by the delimiter (tab), and write an insert procedure, although it could be long with 1 G of data.
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1