Parsing a string into tokens (was: Splitting string into array)

  • (7 Pages)
  • +
  • 1
  • 2
  • 3
  • Last »

94 Replies - 8277 Views - Last Post: 19 August 2018 - 02:49 PM Rate Topic: **--- 3 Votes

#1 Cambalinho   User is offline

  • D.I.C Addict

Reputation: 0
  • View blog
  • Posts: 755
  • Joined: 04-March 12

Parsing a string into tokens (was: Splitting string into array)

Posted 01 July 2018 - 02:24 PM

how can i split a string inclued the delimiter?
sample:
"hello world!!!"
output:
hello
world
!
!
!
and another sample that uses aspes:
""hello world!!!""
ouput:
"hello world!!!"
because the delimiter is the aspes.
how can split them?

This post has been edited by Skydiver: 11 July 2018 - 04:59 AM
Reason for edit:: Changed topic from: "how can i split a string inclued the delimiter?"

Is This A Good Question/Topic? 0
  • +

Replies To: Parsing a string into tokens (was: Splitting string into array)

#2 modi123_1   User is offline

  • Suitor #2
  • member icon



Reputation: 14425
  • View blog
  • Posts: 57,820
  • Joined: 12-June 08

Re: Parsing a string into tokens (was: Splitting string into array)

Posted 01 July 2018 - 03:29 PM

What is your delimiter you are using?
"Aspes"?
Was This Post Helpful? 1
  • +
  • -

#3 Cambalinho   User is offline

  • D.I.C Addict

Reputation: 0
  • View blog
  • Posts: 755
  • Joined: 04-March 12

Re: Parsing a string into tokens (was: Splitting string into array)

Posted 02 July 2018 - 01:39 PM

ok.. let go start ;)
how can i split a string using the space and dot(just for test)?
but the dot must be splitted too and not removed.
i did these code but isn't working :(
std::vector<std::string> splitString(  std::string s, char *delimiter)
{
    std::vector<std::string> ret;
    size_t start = 0;
    size_t end = 0;
    size_t len = 0;
    std::string token;
    for(int i=0; i<s.length(); i++)
    {
        for (int a=0; a<50; a++)
        {

            if(s[i]==delimiter[a] && start==0)
            {
                start=i;
            }
        }

        for (int a=0; a<50; a++)
        {
            if(s[i]==delimiter[a] && start==0)
            {
                token=s.substr(start,i-start);
                cout <<token << endl;
                start=0;

            }
        }
    }
    return ret;
}

//testing:
char *str=" .";//space and  dot
    vector<string> vecstr=splitString("hello world.", str);
    for (std::vector<string>::const_iterator i = vecstr.begin(); i != vecstr.end(); ++i)
        cout << *i << endl;

nothing printed, so my code is wrong
Was This Post Helpful? 0
  • +
  • -

#4 modi123_1   User is offline

  • Suitor #2
  • member icon



Reputation: 14425
  • View blog
  • Posts: 57,820
  • Joined: 12-June 08

Re: Parsing a string into tokens (was: Splitting string into array)

Posted 02 July 2018 - 01:42 PM

Is this homework where you are not supposed to use 'strtok'?
Was This Post Helpful? 1
  • +
  • -

#5 Cambalinho   User is offline

  • D.I.C Addict

Reputation: 0
  • View blog
  • Posts: 755
  • Joined: 04-March 12

Re: Parsing a string into tokens (was: Splitting string into array)

Posted 02 July 2018 - 01:48 PM

i don't know use the 'strtok', but if i use it, the dot is splited too?
Was This Post Helpful? 0
  • +
  • -

#6 modi123_1   User is offline

  • Suitor #2
  • member icon



Reputation: 14425
  • View blog
  • Posts: 57,820
  • Joined: 12-June 08

Re: Parsing a string into tokens (was: Splitting string into array)

Posted 02 July 2018 - 01:55 PM

Well, yeah. That's the whole part about split. It's not "Split but keep the delimiter as the end of the previous token" or "split and keep the delimiter at the head of the secondary token". ;)
http://www.cplusplus...cstring/strtok/

If you want to build your own then determine how it is going to work. Do you put the delimiter on their own token? Start the next token? End the previous?

Example .. split this by the space..

"abc 123 nut"

Normally that would generate an array of

"abc"
"123"
"nut"

.. but do you want that to be

"abc"
" "
"123"
" "
"nut"

.. or
"abc "
"123 "
"nut"

... or

"abc"
" 123"
" nut"
Was This Post Helpful? 1
  • +
  • -

#7 Cambalinho   User is offline

  • D.I.C Addict

Reputation: 0
  • View blog
  • Posts: 755
  • Joined: 04-March 12

Re: Parsing a string into tokens (was: Splitting string into array)

Posted 02 July 2018 - 02:04 PM

from:
"abc 123 nut"
the output is:
"abc"
"123"
"nut"

but from:
"abc 123 nut." (see the dot)
the output:
"abc"
"123"
"nut"
"." (see the dot)
only the spaces are removed

using these sample:
char str[] ="- This, a sample string.";
    char * pch;
    printf ("Splitting string \"%s\" into tokens:\n",str);
    pch = strtok (str," ,.-");
    while (pch != NULL)
    {
        printf ("%s\n",pch);
        pch = strtok (NULL, " ,.-");
    }

the dot, '-' and ',' are removed :(
Was This Post Helpful? 0
  • +
  • -

#8 modi123_1   User is offline

  • Suitor #2
  • member icon



Reputation: 14425
  • View blog
  • Posts: 57,820
  • Joined: 12-June 08

Re: Parsing a string into tokens (was: Splitting string into array)

Posted 02 July 2018 - 02:06 PM

I get that. THe implied issue is you would not.. NOT.. use the period as a delimiter. Instead you would, perhaps, tokenize the string around the space, and use a for loop to go through the characters with 'isalpha'.. if it is not an alpha pop it to it's own token by removing it from one and creating a new.

http://www.cplusplus...locale/isalpha/
Was This Post Helpful? 1
  • +
  • -

#9 Cambalinho   User is offline

  • D.I.C Addict

Reputation: 0
  • View blog
  • Posts: 755
  • Joined: 04-March 12

Re: Parsing a string into tokens (was: Splitting string into array)

Posted 02 July 2018 - 02:10 PM

in these case, the best is use char* instead string?

This post has been edited by Cambalinho: 02 July 2018 - 02:10 PM

Was This Post Helpful? 0
  • +
  • -

#10 modi123_1   User is offline

  • Suitor #2
  • member icon



Reputation: 14425
  • View blog
  • Posts: 57,820
  • Joined: 12-June 08

Re: Parsing a string into tokens (was: Splitting string into array)

Posted 02 July 2018 - 02:15 PM

Use what you think is best.
Was This Post Helpful? 1
  • +
  • -

#11 jimblumberg   User is online

  • member icon

Reputation: 5569
  • View blog
  • Posts: 17,216
  • Joined: 25-December 09

Re: Parsing a string into tokens (was: Splitting string into array)

Posted 02 July 2018 - 02:23 PM

It looks like you're probably accessing one of your arrays out of bounds in the following snippet.

    for(int i=0; i<s.length(); i++)
    {
        for (int a=0; a<50; a++)
        {

            if(s[i]==delimiter[a] && start==0)
            {
                start=i;
            }
        }


Why are you using a C-string for your "delimiters" instead of either a std::vector<char> or a std::string?


Quote

i don't know use the 'strtok', but if i use it, the dot is splited too?

You're using a std::string, why in the world would you even consider strtok()? Just stick with std::string and the many functions available with that class.

Jim
Was This Post Helpful? 1
  • +
  • -

#12 Cambalinho   User is offline

  • D.I.C Addict

Reputation: 0
  • View blog
  • Posts: 755
  • Joined: 04-March 12

Re: Parsing a string into tokens (was: Splitting string into array)

Posted 02 July 2018 - 02:27 PM

i'm trying put the function working, but, for now, without sucess :(
Was This Post Helpful? 0
  • +
  • -

#13 Skydiver   User is online

  • Code herder
  • member icon

Reputation: 6451
  • View blog
  • Posts: 22,083
  • Joined: 05-May 12

Re: Parsing a string into tokens (was: Splitting string into array)

Posted 02 July 2018 - 02:37 PM

It's a matter of judicious use of find_first_of(), find_last_of(), find_first_not_of(), find_last_not_of() and substr.
Was This Post Helpful? 2
  • +
  • -

#14 jimblumberg   User is online

  • member icon

Reputation: 5569
  • View blog
  • Posts: 17,216
  • Joined: 25-December 09

Re: Parsing a string into tokens (was: Splitting string into array)

Posted 02 July 2018 - 03:14 PM

Quote

i'm trying put the function working, but, for now, without sucess

That's probably because you always seem to try to use the hard way to solve your problems. As I said stick with C++ strings and avoid at all costs things like raw arrays (or C-strings) and especially avoid strtok(). IMO, the strtok() function is very complicated, especially for someone not familiar with it's quirks, and you would be better off sticking with strings and the many std::string member functions to do this job. Starting with getline, find, find_first_of, etc.


Jim
Was This Post Helpful? 1
  • +
  • -

#15 CTphpnwb   User is online

  • D.I.C Lover
  • member icon

Reputation: 3802
  • View blog
  • Posts: 13,800
  • Joined: 08-August 08

Re: Parsing a string into tokens (was: Splitting string into array)

Posted 02 July 2018 - 07:11 PM

Using C++ strings and str.find() you could split the string into a vector, then tack the delimiter onto the end of each element but the last one.
Was This Post Helpful? 1
  • +
  • -

  • (7 Pages)
  • +
  • 1
  • 2
  • 3
  • Last »