14 Replies - 7944 Views - Last Post: 16 June 2011 - 02:15 PM

#1 ishkabible  Icon User is offline

  • spelling expret
  • member icon





Reputation: 1659
  • View blog
  • Posts: 5,791
  • Joined: 03-August 09

How does this work?

Posted 12 June 2011 - 03:31 PM

ok so i am trying to figure out how LuaBind works, Boost Python works dose the same thing. i can't figure out how functions are manipulated the way they are, it just makes no sense.

take this.
module(L)
[
    def("sin", &std::sin)
];



this somehow allows Lua to call std::sin and somehow it gets the arguments right and everything. i can't seem to figure it out.

i whipped up a minimal case example(ended up being 160 LOC), this example shows how a function can be wrapped to be used in a language like Lua. however some how things like LuaBind and Boost Python are automatically wrapping the functions. i can't seem to figure out how there doing it. the only way i can think to do it is to create JITed wrapper functions but i don't think that's how there doing it.

minimal case example
#include <iostream>
#include <string>
#include <map>
#include <stack>
#include <cmath>

class FooState;

typedef void(*Function)(FooState&);

void SinWrapper(FooState&);

struct Object {
	//data
	union {
		float num;
		Function func;
		void* userdata;
	};
	//type of data
	enum {
		ttnum,
		ttfunc,
		ttuserdata
	} type;
	//defualt constructor
	Object() {
		type = ttnum;
		num = 0.0;
	}
	//make an object from a number
	Object(double x) {
		type = ttnum;
		num = x;
	}
	//make an object from user data
	Object(void* x) {
		type = ttuserdata;
		userdata = x;
	}
	//make an object from a function
	Object(Function f) {
		type = ttfunc;
		func = f;
	}
	//make an object from another object
	Object(const Object& o) {
		type = o.type;
		switch(type) {
		case ttnum:
			num = o.num;
			break;
		case ttfunc:
			func = o.func;
			break;
		case ttuserdata:
			userdata = o.userdata;
		}
	}
};

class FooState {
public:
	std::map<std::string, Object> data;
	std::map<void*, Function> metaData;
	std::stack<Object> theStack;
	//sets data stored at 'name' to numerical value 'x'
	void set(std::string name, double x) {
		data[name] = Object(x);
	}
	//sets data stored at 'name' to function 'func'
	void set(std::string name, Function func) {
		data[name] = Object(func);
	}
	//sets data stored at 'name' to userdata 'ud'
	void set(std::string name, void* ud) {
		data[name] = Object(ud);
	}
	//sets data stored at 'name' value at the top of the stack
	void set(std::string name) {
		data[name] = theStack.top();
	}
	//sets a meta function for calling userdata
	void setMetaCall(std::string name, Function func) {
		if(data[name].type == Object::ttuserdata) {
			metaData[data[name].userdata] = func;
		}
	}
	//push a numerical value onto the stack
	void push(double x) {
		theStack.push(Object(x));
	}
	//push a function onto the stack
	void push(Function func) {
		theStack.push(Object(func));
	}
	//push userdata on the stack
	void push(void* ud) {
		theStack.push(Object(ud));
	}
	//get a number from the stack
	double popNum() {
		if(theStack.top().type == Object::ttnum) {
			Object x(theStack.top());
			theStack.pop();
			return x.num;
		}
		return 0.0;
	}
	//get a function from the stack
	Function popFunc() {
		if(theStack.top().type == Object::ttfunc) {
			Object x(theStack.top());
			theStack.pop();
			return x.func;
		}
		return NULL;
	}
	//get userdata from the stack
	void* popData() {
		if(theStack.top().type == Object::ttuserdata) {
			Object x(theStack.top());
			theStack.pop();
			return x.userdata;
		}
		return NULL;
	}
	//call someting
	void call(std::string name) {
		switch(data[name].type) {
		case Object::ttfunc:
			data[name].func(*this);
			break;
		case Object::ttnum:
			break;
		case Object::ttuserdata:
			metaData[data[name].userdata](*this);
		}
	}
};

void SinWrapper(FooState& f) {
	f.push(std::sin(f.popNum()));
}

int main() {
	FooState F;
	F.set("sin", SinWrapper);
	F.push(1.5707963267);
	F.call("sin");
	std::cout<<F.popNum();
	//i want this all to be just this
	/*
	module(F) [
		def("sin", &std::sin)
	];
	F.call("sin");
	*/
    return 0;
}



so rather than this

void SinWrapper(FooState& f) {
	f.push(std::sin(f.popNum()));
}

int main() {
	FooState F;
	F.set("sin", SinWrapper);
	F.push(1.5707963267);
	F.call("sin");
	std::cout<<F.popNum();
	return 0;
}



i want this
int main() {
	FooState F;
	module(F) [
		def("sin", &std::sin)
	];
	F.call("sin");
}



how on earth did they do this? how can the functions argument types be stored? how can the generate a wrapper? it just makes no sense.

This post has been edited by ishkabible: 14 June 2011 - 11:59 AM
Reason for edit:: i forgot to declare the FooState in one of the examples


Is This A Good Question/Topic? 0
  • +

Replies To: How does this work?

#2 Bench  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 858
  • View blog
  • Posts: 2,343
  • Joined: 20-August 07

Re: How does this work?

Posted 14 June 2011 - 11:43 AM

Perhaps def is a function which creates an object

Possibly module is a #define macro which declares a variable F whose type has an overloaded operator[] which accepts the object as an argument

All of the symbols used are at least legal in some parts of the language, there's almost certainly a way to abuse C++ to end up with syntax like that.. Whether or not its really desirable I'm not sure :)

Here's one way to abuse it (Note: just to show that its syntactically possible, I really do not condone it! :-) )
#include <string>
#include <map>
#include <cmath>

#define module(FN) functor_type FN; FN

typedef double (*arithmetic_fn_t)(double);
typedef std::pair<std::string, arithmetic_fn_t> association_t;

struct functor_type
{
    std::map<std::string, arithmetic_fn_t> associations;
    void call(const std::string&) 
    {}
    void operator [] (association_t assoc)
    {
        associations.insert(assoc);
    }
};

association_t def(const std::string& alias, arithmetic_fn_t func)
{
    return std::make_pair(alias, func);
} 


int main()
{
    module(F) [
        def("sin", std::sin)
    ];
    F.call("sin");
} 

This post has been edited by Bench: 14 June 2011 - 11:45 AM

Was This Post Helpful? 0
  • +
  • -

#3 ishkabible  Icon User is offline

  • spelling expret
  • member icon





Reputation: 1659
  • View blog
  • Posts: 5,791
  • Joined: 03-August 09

Re: How does this work?

Posted 14 June 2011 - 11:57 AM

im not really worried with how it's syntactically possible, i can see many ways of making the syntax come about. what i don't understand is how a wrapper function is made. some how or another they have to figure out what each type of each argument is, how is that done? once you figure that out you still have to call the function that can have any number of arguments.

take this, say i have a function that takes a string and a function.
template<typename functype>
void callFunc(FooState&, std::string call, functype function) {
   //some code
}



i can call the function above like so
callFunc(F, "f(1.5707963267)", &std::sin); 


such that the value 1.0(sin of 1.5707963267) will be pushed onto the stack of 'F'
how the hell is that possible? both Boost Python and LuaBind are doing it, but how?

also the FooState just wasn't declared in that one example, there is no need to declare it in the macro.

This post has been edited by ishkabible: 14 June 2011 - 12:00 PM

Was This Post Helpful? 0
  • +
  • -

#4 ccubed  Icon User is offline

  • It's That Guy
  • member icon

Reputation: 163
  • View blog
  • Posts: 1,410
  • Joined: 13-June 08

Re: How does this work?

Posted 15 June 2011 - 07:00 AM

Functions can be referenced in more than one ways too. Don't forget that functions are just objects the same as anything else, they just happen to be defined in what they should do as long as they're given the right pieces.

Point in fact, Ever seen the codebase for a Mud? Probably not, but they convert string representations of functions into a map of pointers to the actual function. Same concept here. it's giving the pointer of the function std::sin, which means it's simply calling the function by the pointer.
Was This Post Helpful? 0
  • +
  • -

#5 ishkabible  Icon User is offline

  • spelling expret
  • member icon





Reputation: 1659
  • View blog
  • Posts: 5,791
  • Joined: 03-August 09

Re: How does this work?

Posted 15 June 2011 - 11:02 AM

yes i understand all of this expect for 1 thing.

Quote

but they convert string representations of functions into a map of pointers to the actual function

how the hell do they do that?

This post has been edited by ishkabible: 15 June 2011 - 11:51 AM

Was This Post Helpful? 0
  • +
  • -

#6 ccubed  Icon User is offline

  • It's That Guy
  • member icon

Reputation: 163
  • View blog
  • Posts: 1,410
  • Joined: 13-June 08

Re: How does this work?

Posted 15 June 2011 - 11:15 AM

View Postishkabible, on 15 June 2011 - 12:02 PM, said:

yes i understand all of this expect for 1 thing.

Quote

but they convert string representations of functions into a map of pointers to the actual function

how the hell do they do that?


This is what a typical line looks like:
Function("look",look)



What happens is that they create a map. look now corresponds to the function look, which gives you the contents and description of a room. It's not actually calling look directly, but the program knows what look is because it's an object.

There's a bit more under the hood and they essentially become function pointers, but the important thing here is that all they're doing is using function pointers. You could also do this by creating 'hooks' in your code. That's a bit more advanced. Most likely, what the luabind code does is use a hook, which is basically an advanced function pointer.

Function Pointers

That should make it more clear.
Was This Post Helpful? 0
  • +
  • -

#7 ishkabible  Icon User is offline

  • spelling expret
  • member icon





Reputation: 1659
  • View blog
  • Posts: 5,791
  • Joined: 03-August 09

Re: How does this work?

Posted 15 June 2011 - 11:59 AM

ok, can you implement a simple example?

try implementing this
prototype:
template<class functype>
double callFunc(std::string, functype func);  


this function should take an argument list separated by commas, like so
"10.0, 3.0, 5.0"
all the arguments will be numerical constants. you don't know how many arguments your function needs to accept however; it could be 1 it could be 20. it then passes these arguments in someway to 'func' so that the appropriate value can be returned.

these forces you to know the number of arguments but that is only half my question. how do you know the type of each? I'll leave that question for later though.

you can say "function pointers are the important part" but that's not the question i am asking. im asking how is it possible to deduce the type of each argument and the number of arguments such that it can be called with arguments supplied by interpreting a string.

This post has been edited by ishkabible: 15 June 2011 - 12:02 PM

Was This Post Helpful? 0
  • +
  • -

#8 ccubed  Icon User is offline

  • It's That Guy
  • member icon

Reputation: 163
  • View blog
  • Posts: 1,410
  • Joined: 13-June 08

Re: How does this work?

Posted 15 June 2011 - 01:50 PM

View Postishkabible, on 15 June 2011 - 12:59 PM, said:

ok, can you implement a simple example?

try implementing this
prototype:
template<class functype>
double callFunc(std::string, functype func);  


this function should take an argument list separated by commas, like so
"10.0, 3.0, 5.0"
all the arguments will be numerical constants. you don't know how many arguments your function needs to accept however; it could be 1 it could be 20. it then passes these arguments in someway to 'func' so that the appropriate value can be returned.


You're asking something else. If all you're wanting is a pointer that points to func, which is a function of some type, then you'd still have to pass the type just like any other template.

double *(callFunc) (std::string,<type>*(func) (arguments))



That should work. You're using templates. That doesn't erase the fact C++ still needs a type for that template. Additionally, you're making this too hard.

Your original example was:
def("sin", &std::sin)  



I guarantee you all they're doing is the same thing that Muds do. The text sin should now refer to &std::sin. It could be as simple as:

#define sin &std::sin



or as complicated as

type ptrtosin*(std::sin) (std::string)



Is that what you're asking?

View Postishkabible, on 15 June 2011 - 12:59 PM, said:

these forces you to know the number of arguments but that is only half my question. how do you know the type of each? I'll leave that question for later though.


Well you pass it when you make the function pointer. There is work going on behind the scenes. To know exactly how luabind does it, you'd need to see source. There's a lot of different ways to do it.

View Postishkabible, on 15 June 2011 - 12:59 PM, said:

you can say "function pointers are the important part" but that's not the question i am asking. im asking how is it possible to deduce the type of each argument and the number of arguments such that it can be called with arguments supplied by interpreting a string.


You write a tailor made function to handle it or you rely on the compiler to typecast effectively.
Was This Post Helpful? 0
  • +
  • -

#9 ishkabible  Icon User is offline

  • spelling expret
  • member icon





Reputation: 1659
  • View blog
  • Posts: 5,791
  • Joined: 03-August 09

Re: How does this work?

Posted 15 June 2011 - 02:57 PM

im not getting my point across very well, i apologize for that. i understand function pointers and their capability's. I'm not struggling with the role function pointers play in this.

take this function
double foo(std::string s, double x, int y) {
    //do somthing with arguments
}


you say i am over complicating this. let me show you what LuaBind can do.
say i bind this with LuaBind. now say i call it in Lua as such.
foo(10, 20, 30.5);

this will call foo in C++ like so
foo("10", 20.0, 30);

now say i do something that isn't valid like passing a table for a number.
foo("jake", {x=10}, 10)

this raises an error at run time saying that no conversion could be made between a lua table and type double. now lets say i pass too few arguments, another error at run time will be raised explaining that too few arguments were passed.

so somehow LuaBind is storing information about the number of arguments and each arguments type. how would this be done? finding the number of arguments a function takes alone sounds impossible to me. how on earth is it done?

This post has been edited by ishkabible: 15 June 2011 - 05:39 PM

Was This Post Helpful? 0
  • +
  • -

#10 ccubed  Icon User is offline

  • It's That Guy
  • member icon

Reputation: 163
  • View blog
  • Posts: 1,410
  • Joined: 13-June 08

Re: How does this work?

Posted 15 June 2011 - 05:11 PM

OH! That makes much more sense.

You don't have to. A function pointer literally runs the function, including the result. An error is a result, so all its doing is showing you the result, which in this case is an error. In the case of Lua, since they're still running it through the Lua Interpreter, Lua, like python, is loosely typed. It can easily typecast between multiple types. But when you put too few in, the lua interpreter returns an error. Give it "jake" and it can't convert it to double, returns an error.

In this case, you don't actually have to worry about it because the lua interpreter is returning the error and they've done a wrapper function that runs your code and outputs errors, if any, from the lua interpreter or just outputs the result.

This is the same as redirecting the output of a system call to std::out or your own variable. That's all they're doing likely. So they're not storing that information, because there's no real way to get at it with code, but rather just letting the lua interpreter handle it. How? Most likely, they create a lua interpreter when the first call to init luabind happens and they keep it open until the call to close luabind. That or they're literally opening a lua interpreter, defining your function and then passing it and looking for errors, but it's really more effective to just create a separate thread for the lua interpreter.

This post has been edited by ccubed: 15 June 2011 - 05:14 PM

Was This Post Helpful? 0
  • +
  • -

#11 ishkabible  Icon User is offline

  • spelling expret
  • member icon





Reputation: 1659
  • View blog
  • Posts: 5,791
  • Joined: 03-August 09

Re: How does this work?

Posted 15 June 2011 - 05:44 PM

i know Lua very well, Lua is not doing this. there are only 2 ways to expose C/C++ functions to Lua

1) use a function pointer to a function of type int(*)(lua_state*) and use the lua stack API to pass values in between
2) do the same thing but instead of making it a C function the object is actually a table in which the "__call" meta-method was set to your function pointer. either way you still need a function pointer of type int(*)(lua_state*)

the interpreter doesn't know to raise an error, instead the lua API provides you with functions to check the type of the value passed. say i wanted to expose that foo function, it would look like this.
int lua_foo(lua_state* L) {
   if(!lua_isstring(L, 1)) {
      lua_pushstring(L, "error message goes here");
      lua_error(L);
   }
   if(!lua_isnumber(L, 2)) {
      lua_pushstring(L, "error message goes here");
      lua_error(L);
   }
   if(!lua_isnumber(L, 3)) {
      lua_pushstring(L, "error message goes here");
      lua_error(L);
   }
   std::string s(lua_tostring(L,1));
   double x = lua_tonumber(L, 2));
   int y = lua_tonumber(L, 3));
   lua_pushnumber(L, foo(s, x, y);
   return 1;
}


then to actually expose the function you have to register it somewhere before you can use it.

lua_register(L, "foo", lua_foo);


some how this function is created by LuaBind. i have no clue how though. you can see why it's nice to be able to boil all that down to just this

modual(L) [
    def("foo", foo)
]


20 lines reduced to 3 o_O you can also chain them by adding another like so becuase the function def returns an object with a method named def.

modual(L) [
    def("foo", foo).
    def("bar", bar)
]


that would be like reducing 40 lines to a mere 4 and that is just for 3 argument functions :blink:

the whole thing is like magic or something, how is it done?

edit:
it should be possible to create the function with the pre-proc if you can figure out the type and number of arguments at compile time.

This post has been edited by ishkabible: 15 June 2011 - 05:55 PM

Was This Post Helpful? 0
  • +
  • -

#12 ccubed  Icon User is offline

  • It's That Guy
  • member icon

Reputation: 163
  • View blog
  • Posts: 1,410
  • Joined: 13-June 08

Re: How does this work?

Posted 16 June 2011 - 05:53 AM

Again, because there's a wrapper built around the error messages already, you just don't see it.

I doubt you can simplify your entire lua_foo function to just that module(L) declaration. Somewhere, you're defining lua_foo. It's an object, so lua can interact with it.

This post has been edited by ccubed: 16 June 2011 - 05:54 AM

Was This Post Helpful? 0
  • +
  • -

#13 Xupicor  Icon User is online

  • Nasal Demon
  • member icon

Reputation: 272
  • View blog
  • Posts: 646
  • Joined: 31-May 11

Re: How does this work?

Posted 16 June 2011 - 06:45 AM

View Postccubed, on 15 June 2011 - 05:11 PM, said:

Lua, like python, is loosely typed

I'm enjoying reading this topic, but I have to put something totally off topic here. Python is strongly typed, so if by "loosely" you mean weak typing, then you're wrong. It sure is dynamically typed, but not weakly. ;)
Sorry for intruding, please continue. ^^
Was This Post Helpful? 0
  • +
  • -

#14 ccubed  Icon User is offline

  • It's That Guy
  • member icon

Reputation: 163
  • View blog
  • Posts: 1,410
  • Joined: 13-June 08

Re: How does this work?

Posted 16 June 2011 - 07:31 AM

View PostXupicor, on 16 June 2011 - 07:45 AM, said:

View Postccubed, on 15 June 2011 - 05:11 PM, said:

Lua, like python, is loosely typed

I'm enjoying reading this topic, but I have to put something totally off topic here. Python is strongly typed, so if by "loosely" you mean weak typing, then you're wrong. It sure is dynamically typed, but not weakly. ;)
Sorry for intruding, please continue. ^^


I say loosely typed to any language that doesn't require me to define the type of an object at initialization.

This post has been edited by ccubed: 16 June 2011 - 07:33 AM

Was This Post Helpful? 0
  • +
  • -

#15 ishkabible  Icon User is offline

  • spelling expret
  • member icon





Reputation: 1659
  • View blog
  • Posts: 5,791
  • Joined: 03-August 09

Re: How does this work?

Posted 16 June 2011 - 02:15 PM

Quote

Again, because there's a wrapper built around the error messages already, you just don't see it.

I doubt you can simplify your entire lua_foo function to just that module(L) declaration. Somewhere, you're defining lua_foo. It's an object, so lua can interact with it.


either I'm over complicating this or your not seeing the same probably I'm seeing. im not asking for theory on this, i want to know how it's done. try implementing it if it's so easy. look at how Lua bind works, see if you can create a simplified example using the above foo function.
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1