2 Replies - 448 Views - Last Post: 13 February 2013 - 11:35 PM Rate Topic: -----

#1 jakev383  Icon User is offline

  • New D.I.C Head

Reputation: 1
  • View blog
  • Posts: 41
  • Joined: 27-May 08

Ideas for putting path into database

Posted 13 February 2013 - 07:46 PM

I'm working on a project to index some data to make searching easier. I want to put the path into a database with the serial as a primary key - my path looks like this:

/mnt/server2/boxes/files/serverA/1234567/2013-02-13-00-11-06
/mnt/server2/boxes/files/serverB/6789012/2013-02-13-00-12-14
/mnt/server2/boxes/files/serverA/3456789/2013-02-13-00-01-01

The serial would be the 6th directory in the path (1234567, 6789012, 3456789).
I have gigs of data already in this format so changing the dir tree already in place would not be feasible.
I can walk the directory and extract the data I want using a Bash script, or any other method available at a Linux prompt. There will be new data added daily, but 99% of the data will stay the same from a day to day basis.

My questions for thoughts are:
Is walking the tree in Bash an efficient way to gather the data to be inserted into the database? Would a PHP CLI method be faster?
Is there a semi-efficient way to insert new data without overwriting any existing data in the database? I would imagine I would need to scan the database to see if an entry exists, inserting the new data if it does not exist but wanted to see if others had some other viewpoints I may not be seeing.
Appreciate the opportunity to have a sounding board, and appreciate any thoughts on how to accomplish this task.
Thanks!

Is This A Good Question/Topic? 0
  • +

Replies To: Ideas for putting path into database

#2 CTphpnwb  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 2927
  • View blog
  • Posts: 10,114
  • Joined: 08-August 08

Re: Ideas for putting path into database

Posted 13 February 2013 - 08:58 PM

Seems like a simple task for PHP with PDO, What have you tried?
Was This Post Helpful? 1
  • +
  • -

#3 Dormilich  Icon User is online

  • 痛覚残留
  • member icon

Reputation: 3521
  • View blog
  • Posts: 10,161
  • Joined: 08-June 10

Re: Ideas for putting path into database

Posted 13 February 2013 - 11:35 PM

View Postjakev383, on 14 February 2013 - 03:46 AM, said:

Is walking the tree in Bash an efficient way to gather the data to be inserted into the database? Would a PHP CLI method be faster?

it depends on how good yo are at bash scripting, esp. when it comes to dealing with DB requests. it might be easier for you though to code it all in PHP. generally, I recommend to stick to one scripting language (preferably the one you’re more comfortable with).


View Postjakev383, on 14 February 2013 - 03:46 AM, said:

Is there a semi-efficient way to insert new data without overwriting any existing data in the database?

sure, proper database constraints. e.g. if you try to re-insert an already existing value marked as primary key or unique, the DB will not insert that value/entry.
Was This Post Helpful? 1
  • +
  • -

Page 1 of 1