11 Replies - 3755 Views - Last Post: 04 July 2011 - 03:45 PM Rate Topic: -----

#1 kevin_911  Icon User is offline

  • D.I.C Head

Reputation: 4
  • View blog
  • Posts: 127
  • Joined: 02-April 09

C++ - Debug Assertion Failed

Posted 04 July 2011 - 10:59 AM

Hi Guys need a bit of assistance. Basically I am in the middle of designing a GUi for a console application
which is a forum crawler written in C++ by using various libraries like libcurl and libxml2. Now this application
was designed by a different person who has left our team. I have carried out all the neccessary adjustments for the
application to run smoothly but to no avail :(

The application runs by connecting to SQL server db whereby reading the forum and thread url request by the user.
It then runs the crawl through the forum and eventually sends the information back to the db.

The error I am getting is a 'Debug Assertion Failed' which pops up during execution as shown in the attached image. According to the author the system works 100% :)

Please advice!

Edit: Also the table where it takes the information from for the location of thread includes, forum name, id, url and threadregex and forumregex. threadregex and forum regex has regular expressions for identification of the thread and forum location.

Thanks

Attached image(s)

  • Attached Image

This post has been edited by kevin_911: 04 July 2011 - 11:12 AM


Is This A Good Question/Topic? 0
  • +

Replies To: C++ - Debug Assertion Failed

#2 gabehabe  Icon User is offline

  • GabehabeSwamp
  • member icon




Reputation: 1377
  • View blog
  • Posts: 10,951
  • Joined: 06-February 08

Re: C++ - Debug Assertion Failed

Posted 04 July 2011 - 11:22 AM

Without seeing the code it's hard to judge. I can, however, offer some insight:

Your error is "list iterator is not dereferencable." There are a number of examples that demonstrate that the issue is caused by way of attempting to access a part of a list via an iterator, but technically there is nothing to iterate.

Couple this with the fact that I notice your program is reporting numerous parse errors, I suspect that the application isn't populating the list correctly, then attempting to iterate through the empty list.
Was This Post Helpful? 0
  • +
  • -

#3 jimblumberg  Icon User is offline

  • member icon


Reputation: 3991
  • View blog
  • Posts: 12,315
  • Joined: 25-December 09

Re: C++ - Debug Assertion Failed

Posted 04 July 2011 - 11:25 AM

Without seeing the code that is causing the error it is hard to see what is going on.
Have you run the program through your debugger it should show where the error is detected.

Jim
Was This Post Helpful? 0
  • +
  • -

#4 kevin_911  Icon User is offline

  • D.I.C Head

Reputation: 4
  • View blog
  • Posts: 127
  • Joined: 02-April 09

Re: C++ - Debug Assertion Failed

Posted 04 July 2011 - 12:24 PM

View Postgabehabe, on 04 July 2011 - 12:22 PM, said:

Without seeing the code it's hard to judge. I can, however, offer some insight:

Your error is "list iterator is not dereferencable." There are a number of examples that demonstrate that the issue is caused by way of attempting to access a part of a list via an iterator, but technically there is nothing to iterate.

Couple this with the fact that I notice your program is reporting numerous parse errors, I suspect that the application isn't populating the list correctly, then attempting to iterate through the empty list.



View Postjimblumberg, on 04 July 2011 - 12:25 PM, said:

Without seeing the code that is causing the error it is hard to see what is going on.
Have you run the program through your debugger it should show where the error is detected.

Jim


Thanks guys for your instant reply. The program has about 7 header and source files and I am not exactly sure where the error is, so dont know which source file to attach?!?

In the above image if I click 'Retry' it opens a new window where it says 'autocrawl.exe has triggered a breakpoint' and opens stdthrow.cpp.

Ok here is the main file which I think is executed first.

The main table in the db is 'forums' which in my previous post mentioned contains forum name, id, url, threadregex and forum regex:

This is the info I feed into the table for the exe to pick up.

Quote

Forum name: boards.ie
id: 1
url: http://www.boards.ie/vbulletin/
threadRegex: showthread\.php\?p=([0-9]+)
forumRegex: forumdisplay\.php\?f=([0-9]+)


// autocrawler.cpp : Defines the entry point for the console application.
//
#define OTL_ODBC_MSSQL_2008

#include "stdafx.h"
#include <curl/curl.h>
#include <string>
#include <iostream>
#include "html_downloader.h"
#include <libxml/HTMLparser.h>
#include <libxml/tree.h>
#include <list>
#include <regex>
#include <sstream>
#include "pathcount.h"
#include "crawler.h"
#include "otlv4.h"
#include <stdio.h>
#include "helper.h"

#ifdef LIBXML_TREE_ENABLED

using namespace std;


otl_connect db; // connect object

void insertChs(list<int> chs, int id)
{
	list<int> temp;
	temp.assign(chs.begin(), chs.end());
	int index = 1;

	otl_stream i(200, // buffer size
		"INSERT INTO forumpaths VALUES( "
		":f<int>, :f1<int>, :f2<int>)",
		// SELECT statement
		db // connect object
		); 
	// create select stream
	int tempint;
	while(temp.size() > 0)
	{
		tempint = temp.front();
		temp.pop_front();
		i<<index<<tempint<<id;
		index++;
	}
}


list<int> performAnalysis(int id)
{
	list<int> retVal;

	otl_stream i(200, // buffer size
		"select threadUrl from trainingthreads "
		"where fk_forumId>=:f<int> ",
		// SELECT statement
		db // connect object
		); 
	// create select stream
	i<<id;
	
	char tmpurl[200];
	list<list<int>> tempchs;
	list<int> tempint;
	crawler crawl(0, "", "", "", retVal);

	while(!i.eof()){ // while not end-of-data
		i >> tmpurl;
		tempint = crawl.analyse(tmpurl);
		if(tempint.size() > 1)
			tempchs.push_back(tempint);
		//retVal.push_back(tmp);
	}

	retVal = helper::subList(tempchs);

	return retVal;
}

list<int> getCh(int id)
{
	list<int> retVal;

	otl_stream i(200, // buffer size
		"select nodeindex from forumpaths "
		"where fk_forumId>=:f<int> "
		"ORDER BY [order] ASC",
		// SELECT statement
		db // connect object
		); 
	// create select stream
	i<<id;

	int tmp;
	while(!i.eof()){ // while not end-of-data
		i >> tmp;
		retVal.push_back(tmp);
	}

	if(retVal.size() == 0)
	{
		retVal = performAnalysis(id);
		insertChs(retVal,id);
	}

	return retVal;
}


void select()
{ 
	otl_stream i(200, // buffer size
		"select * from forums",
		db // connect object
		); 
	char f1[200];
	int f2;
	char f3[200];
	char forumRegex[200];
	char threadRegex[200];
	list<int> ch;

	while(!i.eof()){ // while not end-of-data
		i>>f1>>f2>>f3>>threadRegex>>forumRegex;
		cout<<"f1="<<f1<<", f2="<<f2<<", f3="<<f3<<endl;
		
		ch = getCh(f2);
		
		crawler c(f2, f3, threadRegex, forumRegex, ch);
		c.crawl();
	}
}

int _tmain(int argc, _TCHAR* argv[])
{
	otl_connect::otl_initialize(); // initialize ODBC environment
	try{
		db.rlogon(helper::getDbConnString().c_str());
		select();
	}
	catch(otl_exception& p){ // intercept OTL exceptions
		cerr<<p.msg<<endl; // print out error message
		cerr<<p.code<<endl; // print out error code
		cerr<<p.var_info<<endl; // print out the variable that caused the error
		cerr<<p.sqlstate<<endl; // print out SQLSTATE message
		cerr<<p.stm_text<<endl; // print out SQL that caused the error
	}
	db.logoff(); // disconnect from the data source

	string result;
	cin >> result;
				
	return 0;
}
#else
int main(void) {
	fprintf(stderr, "Tree support not compiled in\n");
	exit(1);
}
#endif



This is the second source file which fetches the data from the forum:

#include "htmlParser.h"
#include "helper.h"
#include <iostream>
#define OTL_ODBC_MSSQL_2008

#include "stdafx.h"
#include <curl/curl.h>
#include <string>
#include <iostream>
#include "html_downloader.h"
#include <libxml/HTMLparser.h>
#include <libxml/tree.h>
#include <list>
#include <regex>
#include <sstream>
#include "pathcount.h"
#include "crawler.h"
#include "otlv4.h"
#include <stdio.h>

int crawler::sel(string url)
{ 
	otl_stream o(1, // buffer size should be equal to 1 in case of stored procedure call
		"{call GetThreadId("
		" :forumId<int,in>, "
		" :B<char[200],in>, "
		" :C<int,inout> "
		")}",
		// stored procedure call
		db // connect object
		);
	
	o<<id<<url.c_str()<<0;

	int a;

	o>>a;
	cout<<"A="<<a<<endl;
	return a;
}

crawler::crawler(int i, std::string url, std::string threadRx, std::string forumRx, list<int> ch)
{
	//add url to list, depending on regex
	forumurls.push_back(url);
	threadRegex = threadRx;
	forumRegex = forumRx;
	children = ch;
	urlExp = "[A-Za-z]+://>/[A-Za-z.]+";
	id = i;	
}

crawler::~crawler(void)
{
}

list<int> crawler::min_analyse(string url, list<int> ch)
{
	html_downloader h(url);
	string result = h.getHtml();

	htmlParser hp(result, ch, true);

	return hp.min_analyse();
}

list<int> crawler::analyse(string url)
{
	html_downloader h(url);
	string result = h.getHtml();

	list<int> ch;

	htmlParser hp(result, ch, true);

	ch = hp.analyse();

	return ch;
}

void crawler::fetch(string url, bool thread)
{
	html_downloader h(url);
	string result = h.getHtml();

	htmlParser hp(result, children, thread);

	int threadId = 0;

	if(thread)
	{
		otl_connect::otl_initialize(); // initialize ODBC environment
		try{
			db.rlogon(helper::getDbConnString().c_str());
			//stored_proc(); // invoking stored procedure
			threadId = sel(url);
		}
		catch(otl_exception& p){ // intercept OTL exceptions
			cerr<<p.msg<<endl; // print out error message
			cerr<<p.code<<endl; // print out error code
			cerr<<p.var_info<<endl; // print out the variable that caused the error
			cerr<<p.sqlstate<<endl; // print out SQLSTATE message
			cerr<<p.stm_text<<endl; // print out SQL that caused the error
		}
		db.logoff(); // disconnect from the data source
	}


	list<xmlChar*> urls = hp.parseHtml(threadId);

	list<xmlChar*>::iterator i;

	list<string> absurls;
	string domain = helper::firstMatch(url, urlExp);

	int qz = 0;

	for(i=urls.begin(); i != urls.end(); ++i) 
	{
		string temp = (string)(char*)*i;

		if(!helper::contains(temp, urlExp))
		{
			if(temp.size() == 0)
			{
				temp = domain;
			}
			else
			{
				if(temp[0]=='/')
					temp = domain + temp;
				else
					temp = domain + '/' + temp;
			}
		}
		//disabling crawling
		absurls.push_back(temp);
	}

	list<string>::iterator s;
	list<string> localurls;

	for(s=absurls.begin(); s != absurls.end(); ++s)
		if(domain.compare(((string)*s).substr(0,domain.length()))==0)
			localurls.push_back(*s);

	for(s=localurls.begin(); s != localurls.end(); ++s)
	{	
		if(helper::contains(*s, threadRegex))
		{
			if(!helper::inStrList(threadurls, *s))
			{
				if(!helper::inStrList(visited, *s))
				{
					threadurls.push_back(*s);
				}
			}
		}
		else if(helper::contains(*s, forumRegex))
		{
			if(!helper::inStrList(forumurls, *s))
			{
				if(!helper::inStrList(visited, *s))
				{
					forumurls.push_back(*s);
				}
			}
		}
	}

	visited.push_back(url);
}

void crawler::crawl(void)
{
	while((forumurls.size() > 0 || threadurls.size() > 0) && visited.size() < 10000)
	{
		if(forumurls.size() > 0)
		{	
			string t = forumurls.front();
			forumurls.pop_front();

			crawler::fetch(t, false);
		}
		if(threadurls.size() > 0)
		{	
			string t = threadurls.front();
			threadurls.pop_front();

			crawler::fetch(t, true);
		}
	}
}

This post has been edited by kevin_911: 04 July 2011 - 12:27 PM

Was This Post Helpful? 0
  • +
  • -

#5 Oler1s  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 1395
  • View blog
  • Posts: 3,884
  • Joined: 04-June 09

Re: C++ - Debug Assertion Failed

Posted 04 July 2011 - 12:51 PM

> In the above image if I click 'Retry' it opens a new window where it says 'autocrawl.exe has triggered a breakpoint' and opens stdthrow.cpp.

Walk up the callstack please. It should be on the bottom right hand side of the screen or somewhere in the bottom where you can click up the callstack. Go up until you find your code.
Was This Post Helpful? 0
  • +
  • -

#6 kevin_911  Icon User is offline

  • D.I.C Head

Reputation: 4
  • View blog
  • Posts: 127
  • Joined: 02-April 09

Re: C++ - Debug Assertion Failed

Posted 04 July 2011 - 02:07 PM

View PostOler1s, on 04 July 2011 - 01:51 PM, said:

> In the above image if I click 'Retry' it opens a new window where it says 'autocrawl.exe has triggered a breakpoint' and opens stdthrow.cpp.

Walk up the callstack please. It should be on the bottom right hand side of the screen or somewhere in the bottom where you can click up the callstack. Go up until you find your code.


Hi 'Oler1s' image attached below shows the result of my call stack.

Attached image(s)

  • Attached Image

Was This Post Helpful? 0
  • +
  • -

#7 Oler1s  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 1395
  • View blog
  • Posts: 3,884
  • Joined: 04-June 09

Re: C++ - Debug Assertion Failed

Posted 04 July 2011 - 02:19 PM

Ok, so you have an image there, but so what?

You need to actually go up the callstack until you find your code. Looking at that list, it's helper::subList, right? So click on that item in the list to go to your code, and you can see the lines of code that are problematic.

Look, your problem is that your program is failing, and you have no idea where in the code to start looking. The debugger is telling you exactly where the error came from, so that's a starting point. That's what I'm telling you to do.
Was This Post Helpful? 0
  • +
  • -

#8 kevin_911  Icon User is offline

  • D.I.C Head

Reputation: 4
  • View blog
  • Posts: 127
  • Joined: 02-April 09

Re: C++ - Debug Assertion Failed

Posted 04 July 2011 - 02:42 PM

View PostOler1s, on 04 July 2011 - 03:19 PM, said:

Ok, so you have an image there, but so what?

You need to actually go up the callstack until you find your code. Looking at that list, it's helper::subList, right? So click on that item in the list to go to your code, and you can see the lines of code that are problematic.

Look, your problem is that your program is failing, and you have no idea where in the code to start looking. The debugger is telling you exactly where the error came from, so that's a starting point. That's what I'm telling you to do.


Sorry Oler1s for being a noob :oops: The author has told me the code works perfectly but on my PC it is failing so dunno who is wrong, if you know what I mean :)

I have been trying to understand the code but not able to pinpoint the problem. Dunno if it is my input into the db which wrong or the code?

But anyhow, yes helper.cpp is one of the function while used by the system. If clicked on it takes me to helper.cpp line number 29 as shown below.

The helper function, basically carries out the following:

1. GetLinkHref: This function aids in the extraction of a link from an anchor element in the HTML

2. SubList: Takes a list of lists of integers, and compares them to find the largest common subset of valuues that bein the list.

3. FirstMatch: Used to simplify the use of a regular expression by excepting a string of input, and a string defining the expression.

4. Contains: This is another function to simplify the use of regular expression.

5. InStringList: This function accepts a list of strings, and a string and will return true if the string is contained in the list and false if it isn't.

6. StringList: This function accepts a node, and for each child element in that node it concatenates the contents of that node and all its childern into a string, and adds the string to a list then returns the list.

7. StringRex: This function recursively converts the content of a node and all its childern into a string and return the string.

#include "helper.h"
#include <stdio.h>
#include <locale>
#include <regex>
#include <sstream>


helper::helper(void)
{
}


helper::~helper(void)
{
}
xmlChar* helper::getLinkHref(xmlNode *a_node)
{
	xmlAttr *cur_node = NULL;

	for(cur_node = a_node->properties; cur_node; cur_node = cur_node->next)
	{
		if(xmlStrEqual(cur_node->name, xmlCharStrdup("href"))==1 && cur_node->children != NULL)
			{
				return (cur_node->children->content);
			}
	}
}

list<int> helper::subList(list<list<int>> nums)
{
	list<int> temp1 = nums.front();
	nums.pop_front();
	list<int> retVal;
	retVal.assign(temp1.begin(), temp1.end());
	list<int> temp2;
	list<int> temp3;

	while(nums.size() > 0)
	{
		temp1 = nums.front();
		nums.pop_front();

		temp2.assign(retVal.begin(), retVal.end());

		temp3.clear();

		while(temp1.size()>0 && temp2.size() >0)
		{
			int a = temp1.front();
			temp1.pop_front();

			int b = temp2.front();
			temp2.pop_front();

			if(a==B)/>
			{
				temp3.push_back(a);
			}
			else
			{
				break;
			}
		}

		retVal.assign(temp3.begin(), temp3.end());
	}
	return retVal;
}

string helper::firstMatch(string input, string exp)
{
	std::tr1::cmatch res;
    std::tr1::regex rx(exp);
    std::tr1::regex_search(input.c_str(), res, rx);
	return res[0].str();
}

bool helper::contains(string input, string exp)
{
	std::tr1::regex rx(exp);
	return regex_search(input.begin(), input.end(), rx);
}

string helper::intToString(int a)
{
	std::stringstream ss;
	std::string s;
	ss<<a;
	s = ss.str();
	return s;
}

bool helper::inStrList(list<string> l, string s)
{
	list<string>::iterator i;

	for(i=l.begin(); i != l.end(); ++i)
		if(s.compare(*i)==0)
			return true;
	return false;
}

list<string> helper::stringList(xmlNode * a_node)
{
	xmlNode *cur_node = NULL;
	list<string> retVal;

	for (cur_node = a_node; cur_node; cur_node = cur_node->next) {
        if (cur_node->type == XML_ELEMENT_NODE) {
			if(cur_node->children != NULL)
			{
				string s = getStringRec(cur_node->children);

				retVal.push_back(s);
			}
        }
    }

	return retVal;
}

string helper::getStringRec(xmlNode * a_node)
{
	xmlNode *cur_node = NULL;

	string temp = "";

	for (cur_node = a_node; cur_node; cur_node = cur_node->next) {
        if (cur_node->type == XML_ELEMENT_NODE) {
			if(cur_node->children != NULL && cur_node->children->content != NULL)
			{
				std::stringstream ss;
				std::string s;
				ss<<cur_node->children->content;
				s = ss.str();
				char c;
				locale loc;

				bool nonws = false;
				int i;
				for(i=0;i<s.length();i++)
				{
					c=s[i];
					if (!isspace(c,loc))
					{
						nonws = true;
					}
				}

				if(nonws)
				{
					temp.append(s);
				}
			}			
			if(cur_node->children != NULL)
			{
				temp.append(getStringRec(cur_node->children));
			}
        }
    }

	return temp;
}

string helper::getDbConnString()
{
	return "uid=tester;pwd=******;dsn=crawldb";
}

This post has been edited by kevin_911: 04 July 2011 - 02:44 PM

Was This Post Helpful? 0
  • +
  • -

#9 Karel-Lodewijk  Icon User is offline

  • D.I.C Addict
  • member icon

Reputation: 449
  • View blog
  • Posts: 849
  • Joined: 17-March 11

Re: C++ - Debug Assertion Failed

Posted 04 July 2011 - 03:03 PM

pop_front() would segfault if the list was empty, is it ?
Was This Post Helpful? 1
  • +
  • -

#10 kevin_911  Icon User is offline

  • D.I.C Head

Reputation: 4
  • View blog
  • Posts: 127
  • Joined: 02-April 09

Re: C++ - Debug Assertion Failed

Posted 04 July 2011 - 03:13 PM

View PostKarel-Lodewijk, on 04 July 2011 - 04:03 PM, said:

pop_front() would segfault if the list was empty, is it ?


I am sorry but how I can check this list?
Was This Post Helpful? 0
  • +
  • -

#11 Oler1s  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 1395
  • View blog
  • Posts: 3,884
  • Joined: 04-June 09

Re: C++ - Debug Assertion Failed

Posted 04 July 2011 - 03:16 PM

Quote

The author has told me the code works perfectly but on my PC it is failing so dunno who is wrong
The author is. If he's still responsible for making sure the program works, you have every right to demand it of him. "It worked on my computer" or "It worked before" are not acceptable excuses.

The function in question is subList, so you should look at that function for now, as opposed to other functions like contains, right?

If you look at the callstack, you can see that the error originates on a call to front(). In the subList function, I see the first line of code calling nums.front. This requires that the list be populated, as you can't get the first item of an empty list. Since there's no error checking here, it has be done elsewhere up stack at some point.

Looking at the callstack, sublist is called by performAnalysis, and you posted code for that. I don't see any code that checks the size of the list. Since that list is being created in performAnalysis, and neither performAnalysis, nor subList ensures that the list has any data. So like gabehabe guessed before, it's quite possible that the error is because the code is operating on an empty list, but assumes there is actually something.

You can verify this if you actually put up a debug point (I would put it in perform analysis). Step through with a debugger and see the behavior for yourself.

EDIT: You can inspect variables as you step through code. While stepping through, if you hover over a variable, or click it, or right click or something, you should be able to either get a popup with the value or add it to a watch list. I believe the list is by default on the bottom left of the screen.

This post has been edited by Oler1s: 04 July 2011 - 03:18 PM

Was This Post Helpful? 1
  • +
  • -

#12 kevin_911  Icon User is offline

  • D.I.C Head

Reputation: 4
  • View blog
  • Posts: 127
  • Joined: 02-April 09

Re: C++ - Debug Assertion Failed

Posted 04 July 2011 - 03:45 PM

Thanks guys for your help. I am going to forward this to the author somehow and see what
has to be said about it. Likewise I will update this thread accordingly.
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1