Word sorting program

  • (10 Pages)
  • +
  • 1
  • 2
  • 3
  • 4
  • Last »

140 Replies - 7113 Views - Last Post: 04 November 2013 - 03:14 PM Rate Topic: -----

#16 lewm  Icon User is offline

  • D.I.C Head

Reputation: 7
  • View blog
  • Posts: 160
  • Joined: 29-March 13

Re: Word sorting program

Posted 03 September 2013 - 09:18 AM

Hi guys, ok i have re-wrote the program (without cheating and looking at a solution), it works for a few hundred words but when i use my full list 200,000 words it crashes, heres the code: (i know my error handling is none existant...)
#include <stdlib.h>
#include <stdio.h>
#include <windows.h>
#include <math.h>
#include <string.h>
#include <dos.h>
#include <time.h>

long int count_words(void);

int main(void)
{
    FILE *p, *sorted;
    long int total_words=0, i=0, j=0, cpy=0;

    total_words=count_words();
    char word_array[total_words+1][50];///It crashes on this line///
    word_array[total_words][0]='*';
    p=fopen("words","r");
    do
    {
        fscanf(p,"%s",&word_array[i][0]);
        i++;
    }while(i<total_words);
    fclose(p);
    i=0;
    sorted=fopen("sorted","w");
    for(i=0;i<total_words;i++)
    {
        for(j=i+1;j<total_words;j++)
        {
            if(strcmp(word_array[i],word_array[j])==0)
            {
                cpy=1;
            }
        }
        if(!cpy)
        {
            fprintf(sorted,"%s\n",word_array[i]);
        }
        cpy=0;
    }
    fprintf(sorted,"*");
    fclose(sorted);

    return 0;
}

long int count_words(void)
{
    FILE *p;
    long int word_count=0;
    char word[100]={0};

    p=fopen("words","r");
    do
    {
        fscanf(p,"%s",&word);
        if(word[0]!='*')
            word_count++;
    }while(word[0]!='*');
    fclose(p);
    return(word_count);
}

This post has been edited by lewm: 03 September 2013 - 09:20 AM

Was This Post Helpful? 0
  • +
  • -

#17 #define  Icon User is offline

  • Duke of Err
  • member icon

Reputation: 1344
  • View blog
  • Posts: 4,613
  • Joined: 19-February 09

Re: Word sorting program

Posted 03 September 2013 - 10:32 AM

This is really C++ code :

16	    total_words=count_words();
17	    char word_array[total_words+1][50];///It crashes on this line///




The array is probably to big.

You could create an array of char pointers, and allocate storage for each string.
Was This Post Helpful? 1
  • +
  • -

#18 lewm  Icon User is offline

  • D.I.C Head

Reputation: 7
  • View blog
  • Posts: 160
  • Joined: 29-March 13

Re: Word sorting program

Posted 03 September 2013 - 10:43 AM

An array of char pointers? Can you help me understand how to do that and how it works please?
Was This Post Helpful? 0
  • +
  • -

#19 #define  Icon User is offline

  • Duke of Err
  • member icon

Reputation: 1344
  • View blog
  • Posts: 4,613
  • Joined: 19-February 09

Re: Word sorting program

Posted 03 September 2013 - 11:27 AM

You maybe aware that a char pointer (char*) can be used to point to (hold address of) an array of char (possibly a C type string).

  char str[] = "hello string";
  char* ptr;
  ptr = str;

  printf("%s", ptr);




You can create a list of pointers:

  char* str_list[50];



and allocate memory for each string.

malloc
Was This Post Helpful? 1
  • +
  • -

#20 lewm  Icon User is offline

  • D.I.C Head

Reputation: 7
  • View blog
  • Posts: 160
  • Joined: 29-March 13

Re: Word sorting program

Posted 03 September 2013 - 11:32 AM

View Post#define, on 03 September 2013 - 11:27 AM, said:

You maybe aware that a char pointer (char*) can be used to point to (hold address of) an array of char (possibly a C type string).

  char str[] = "hello string";
  char* ptr;
  ptr = str;

  printf("%s", ptr);




You can create a list of pointers:

  char* str_list[50];



and allocate memory for each string.

malloc

Is (char* p) the same as (char *p)?
Was This Post Helpful? 0
  • +
  • -

#21 vividexstance  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 659
  • View blog
  • Posts: 2,270
  • Joined: 31-December 10

Re: Word sorting program

Posted 03 September 2013 - 11:38 AM

Yes, it's a matter of style. Some people like the '*' next to the variable name so you write something like this:
char *str1, *str2;


Was This Post Helpful? 1
  • +
  • -

#22 lewm  Icon User is offline

  • D.I.C Head

Reputation: 7
  • View blog
  • Posts: 160
  • Joined: 29-March 13

Re: Word sorting program

Posted 03 September 2013 - 11:57 AM

View Post#define, on 03 September 2013 - 11:27 AM, said:

You maybe aware that a char pointer (char*) can be used to point to (hold address of) an array of char (possibly a C type string).

  char str[] = "hello string";
  char* ptr;
  ptr = str;

  printf("%s", ptr);




You can create a list of pointers:

  char* str_list[50];



and allocate memory for each string.

malloc

ok ive used malloc before but i dont know how to allocate memory to this array of pointers ive created.

View Postvividexstance, on 03 September 2013 - 11:38 AM, said:

Yes, it's a matter of style. Some people like the '*' next to the variable name so you write something like this:
char *str1, *str2;


if i do (char* a, b, c) are all these now pointers or just (a)?
Was This Post Helpful? 0
  • +
  • -

#23 vividexstance  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 659
  • View blog
  • Posts: 2,270
  • Joined: 31-December 10

Re: Word sorting program

Posted 03 September 2013 - 11:58 AM

Just 'a' would be a pointer, the other two, 'b' and 'c', are just regular chars.
Was This Post Helpful? 1
  • +
  • -

#24 Adak  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 331
  • View blog
  • Posts: 1,168
  • Joined: 01-April 11

Re: Word sorting program

Posted 03 September 2013 - 04:17 PM

Several suggestions for you:

Try these two things to ease your memory problem:

1) Make your word array, global (declared above main(). An array INSIDE a function, (a regular static array like you have been using), can only access about 2MB, because it can't use the larger memory heap.

But a global array, will use heap memory - so put it to good use. Note that arrays you dynamically allocate also use heap memory, but a global array is simpler.

2) The longest "regular" word (non-scientific, not a word made up just to be longest), is antidisestablishmentarianism, which has 28 letters (so needs only 29 spaces. So change your word length to 29, instead of 50. Unless you are including scientific (chemistry) or "funny" words.

You may not make it to 200,000, but you will make it to a much larger number than you can with your current code.

A good way to eliminate duplicate words, is to sort them, and then go through the entire sorted array of words. If words[i], does not equal words[i+j], then print out the word. Otherwise, don't.

I'll have more info for you, later. Try this - and then try the array of pointers.

Post up how it works out. I have more info for you, later on.

This post has been edited by Adak: 03 September 2013 - 04:18 PM

Was This Post Helpful? 1
  • +
  • -

#25 jjl  Icon User is offline

  • Engineer
  • member icon

Reputation: 1074
  • View blog
  • Posts: 4,533
  • Joined: 09-June 09

Re: Word sorting program

Posted 03 September 2013 - 04:39 PM

Quote

1) Make your word array, global (declared above main(). An array INSIDE a function, (a regular static array like you have been using), can only access about 2MB, because it can't use the larger memory heap.


You should never make global variables just to satify memory requirements. If you can't fit memory on the stack, then allocate it on the heap, but don't make the variable global. The reasons can be found in the link

http://c2.com/cgi/wi...VariablesAreBad
Was This Post Helpful? 1
  • +
  • -

#26 Adak  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 331
  • View blog
  • Posts: 1,168
  • Joined: 01-April 11

Re: Word sorting program

Posted 03 September 2013 - 05:47 PM

jjl, I agree with you except - I was chatting with a programmer who released a program with exactly this kind of requirement, and he had used the very large global array as part of a commercial program, very successfully.

I'm not recommending it for everyone, but for a student who hasn't been taught to allocate memory yet, it deserves consideration.
Was This Post Helpful? 0
  • +
  • -

#27 Skydiver  Icon User is offline

  • Code herder
  • member icon

Reputation: 3574
  • View blog
  • Posts: 11,114
  • Joined: 05-May 12

Re: Word sorting program

Posted 03 September 2013 - 06:17 PM

From what I recall, the original implementation of PKZIP used globals for its hashtables, dictionaries, and other data structures.

Also doesn't the code generated by flex and bison use globals?
Was This Post Helpful? 0
  • +
  • -

#28 lewm  Icon User is offline

  • D.I.C Head

Reputation: 7
  • View blog
  • Posts: 160
  • Joined: 29-March 13

Re: Word sorting program

Posted 04 September 2013 - 11:32 AM

Thanks guys, just for the record im not a student this is a hobby of mine, sad, i know.
Any help on how to allocate memory to the pointer array would be very helpful.
I think i could use a 3 or 4 dimentional array to also solve my problem?? Not sure though as ive never used one.
Was This Post Helpful? 0
  • +
  • -

#29 lewm  Icon User is offline

  • D.I.C Head

Reputation: 7
  • View blog
  • Posts: 160
  • Joined: 29-March 13

Re: Word sorting program

Posted 04 September 2013 - 11:53 AM

Ive just thought how can i make the array global if i dont know its size? Can i still declare it like this:
(char array[variable]) as a global?
Was This Post Helpful? 0
  • +
  • -

#30 jimblumberg  Icon User is online

  • member icon


Reputation: 4070
  • View blog
  • Posts: 12,557
  • Joined: 25-December 09

Re: Word sorting program

Posted 04 September 2013 - 11:58 AM

As long as "variable" is a compile time constant, yes. But otherwise no.

You probably shouldn't be using VLA in the first place. Stick with using compile time constants or dynamic memory allocation with malloc().

Jim
Was This Post Helpful? 1
  • +
  • -

  • (10 Pages)
  • +
  • 1
  • 2
  • 3
  • 4
  • Last »