XML to MySQL big Problem!

weird and big problem

Page 1 of 1

4 Replies - 1226 Views - Last Post: 11 February 2009 - 05:03 AM Rate Topic: -----

#1 hrvoje89  Icon User is offline

  • D.I.C Head

Reputation: 0
  • View blog
  • Posts: 63
  • Joined: 05-February 09

XML to MySQL big Problem!

Post icon  Posted 11 February 2009 - 02:54 AM

Hello!

I have next problem...

i have
$doc->load( 'http://www.elkotech.hr/api/?api_key=xyz&method=elko.items.getItems&sig=xyz' );


And that is XML document with 1000-2000 items.. then i use DOMDocument to load it and read it... the I use foreach loop so i can
store each item into MySQL DB...So i store items in db...

My script is using cron job to run every 45 minutes to update articles in DB (clean DB [truncate] then insert new articles).

So ... it works fine .. but from time to time it inserts 0 articles??? And i know that XML file always has got at least 1000 articles...
then on next update script inserts normal number of articles...

So where is the problem?
could it be the memory/cpu usage?
How can i make the script to use less memory?

Ty!

Is This A Good Question/Topic? 0
  • +

Replies To: XML to MySQL big Problem!

#2 Auzzie  Icon User is offline

  • D.I.C Addict
  • member icon

Reputation: 43
  • View blog
  • Posts: 573
  • Joined: 20-January 09

Re: XML to MySQL big Problem!

Posted 11 February 2009 - 03:07 AM

Well have you checked your error logs? also i have to say you havent shown us any of the code to see if the problem is in that,same with the cron job

[rules][/rules]
Was This Post Helpful? 0
  • +
  • -

#3 hrvoje89  Icon User is offline

  • D.I.C Head

Reputation: 0
  • View blog
  • Posts: 63
  • Joined: 05-February 09

Re: XML to MySQL big Problem!

Posted 11 February 2009 - 03:25 AM

Hehe... it's a lot of code,but ok.. you asked for it :D p.s. this is old and bad coding..but it worked until now :D

<?

$doc = new DOMDocument();
$doc->load( 'http://www.elkotech.hr/api/?api_key=132&method=elko.items.getItems&sig=123' );

$items = $doc->getElementsByTagName( "item" );

$con = mysql_connect('localhost','xyz','xyz');
mysql_select_db('xyz',$con);

$empty_db = mysql_query( 'TRUNCATE TABLE artikli', $con ) or die();

$x = 0;
$dodano_novih_artikala = 0;

foreach ( $items as $item )
{
$dodano_novih_artikala++;
	/* Part Number */
	$part = $item->getAttribute( "partnumber" );
	#Categ
	$categ = $item->getAttribute( "categ" );
	#Group
	$group = $item->getAttribute( "group" );
	#ORIGIN
	$origin = $item->getAttribute( "origin" );
	#InStock
	$stock = $item->getAttribute( "instock" );
	#Price
	$price = $item->getAttribute( "price" );

	#Name
	$name = $item->getElementsByTagName( "name" );
	$naziv_artikla = $name->item( 0 )->nodeValue;
	$clean_name = uncdata( $naziv_artikla );
	$clean_name2 = str_replace( "'", "", $clean_name );

	#Description
	$desc = $item->getElementsByTagName( "description" );
	$opis = $desc->item( 0 )->nodeValue;
	$desc_clean = str_replace( '"', ' ', $opis );
	$desc_clean2 = str_replace( "'", ' ', $desc_clean );
	$desc_clean3 = mysql_real_escape_string( $desc_clean2 );
	#Slika
	$pic = $item->getElementsByTagName( "picture" );
	$slika = $pic->item( 0 )->nodeValue;


	$def = 1;
	 /* Ako = 1 onda src_img == "" */

	if ( preg_match("/gif/i", $slika) )
	{
		$src_img = ".gif";
		$def = 0;
	} else
		if ( preg_match("/jpg/i", $slika) )
		{
			$src_img = ".jpg";
			$def = 0;
		} else
			if ( preg_match("/jpeg/i", $slika) )
			{
				$src_img = ".jpg";
				$def = 0;
			} else
				if ( preg_match("/png/i", $slika) )
				{
					$src_img = ".png";
					$def = 0;
				}

	if ( $def == 1 )
	{
		$src_img = "";
	}

	$price1 = str_replace( '.', '', $price );
	$price2 = str_replace( ',', '.', $price1 );

	$query_check = mysql_query( 'SELECT * FROM artikli WHERE part_num="' . $part .
		'" OR name="' . $clean_name2 . '"', $con ) or die( 'error' . mysql_error() .
		' /*/ ' . mysql_errno() );
	$num = mysql_num_rows( $query_check );

	if ( $num > 0 )
	{
		while ( $red = mysql_fetch_array($query_check) )
		{

			if ( $stock != $red['in_stock'] )
			{

				$query = mysql_query( 'UPDATE artikli SET in_stock="' . $stock .
					'" WHERE part_num="' . $part . '" AND name="' . $clean_name2 . '"', $con ) or
					die( 'error2' );

			}

			if ( $price != $red['price'] )
			{

				$query_p = mysql_query( 'UPDATE artikli SET price="' . $price2 .
					'" WHERE part_num="' . $part . '" AND name="' . $clean_name2 . '"', $con ) or
					die( 'error3' );

			}

		}
	} else
	{

		if ( file_exists('upload/' . $part . $src_img) )
		{
			/* Nista */
		} else
			if ( $src_img != "" )
			{

				if ( $slika != "" )
				{

					$remoteImage = $slika; //full url to image.
					$getFile = file_get_contents( $remoteImage ); //get the remote image into a string.

					$fp = fopen( "upload/" . $part . $src_img, "w" );
					fwrite( $fp, $getFile );
					fclose( $fp );


				}

			}

		if ( filesize("upload/" . $part . $src_img) > 0 && $src_img != "" )
		{

			$query = mysql_query( 'INSERT INTO `artikli` (`id`, `part_num`, `cat_id`, `in_stock`, `price`, `name`, `desc`, `picture`, `img_ex`, `grupa`, `origin`) VALUES ("", "' .
				$part . '", "' . $categ . '", "' . $stock . '", "' . $price2 . '", "' . $clean_name2 .
				'", "' . $desc_clean3 . '", "' . $slika . '", "' . $src_img . '", "' . $group .
				'", "' . $origin . '")', $con ) or die( mysql_error() . mysql_errno() );

		} else
			if ( filesize("upload/" . $part . $src_img) == 0 )
			{

				$query = mysql_query( 'INSERT INTO `artikli` (`id`, `part_num`, `cat_id`, `in_stock`, `price`, `name`, `desc`, `picture`, `img_ex`, `grupa`, `origin`) VALUES ("", "' .
					$part . '", "' . $categ . '", "' . $stock . '", "' . $price2 . '", "' . $clean_name2 .
					'", "' . $desc_clean3 . '", "", "", "' . $group . '", "' . $origin . '")', $con ) or
					die( mysql_error() . mysql_errno() );

			}

		unset( $part_num, $categ, $stock, $price, $name, $clean_name, $clean_name2, $desc,
			$desc_clean, $desc_clean2, $picture, $group, $origin, $clean_part_num1, $clean_part_num2,
			$clean_part_num3, $clean_part_num4, $src_img, $remoteImage, $getFile );


	}
}

$date = date('d.m.Y').' u '.date('H:i');
  $d = mysql_query("INSERT INTO `update` (`date`,`num`)VALUES ('".$date."', '".$dodano_novih_artikala."')",$con) or die(mysql_error());

mysql_free_result($query) or die(mysql_error());
mysql_free_result($d);
mysql_free_result($query_check);

?>

Was This Post Helpful? 0
  • +
  • -

#4 Auzzie  Icon User is offline

  • D.I.C Addict
  • member icon

Reputation: 43
  • View blog
  • Posts: 573
  • Joined: 20-January 09

Re: XML to MySQL big Problem!

Posted 11 February 2009 - 03:33 AM

I'm not much one with DOM, but have you checked your error logs to see if anything there is generated and have you looked at DOMDocument and seeing as your using an XML file thought of looking at XML guides on PHP.net, includes DOMXML ;)
Was This Post Helpful? 0
  • +
  • -

#5 Valek  Icon User is offline

  • The Real Skynet
  • member icon

Reputation: 544
  • View blog
  • Posts: 1,713
  • Joined: 08-November 08

Re: XML to MySQL big Problem!

Posted 11 February 2009 - 05:03 AM

SimpleXML is also very good for parsing XML documents. I use it myself on a large project I'm working on.
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1