[FrozenSnake]

I have a question about "judges".

A judge is:
A judge is the thingy that you upload code to on a website and the judge compile, execute and run the program. If the output from the compiled program is correct it say "Ok" and if not it give a "didn't work".

Question:
How are these made? No source code is wanted just some information on what I can/should read about to gain the knowledge to make one of my own.

Don't know if this is the right section to ask if not please move to the correct section

[Oler1s]

Before I feed out any information, I'm curious as to what you think so far. So you don't know how judges are done, but let's say the idea of an automated judge was completely new. I am going to pay you $1 million to pull it off.

What thoughts do you have? What do you think a judge needs to be able to do? What do you imagine you have to study or learn about?

[FrozenSnake]

My guess is that the judge need to be able to save the code in a file.
Put the path in the compiler where the file to compile are located.
Compile the program, run it and read the output then match it to a file with the result.

The "judge" can probably be made in php/ajax/js that run the selected compiler.

1. User add code.
2. User press "run", code is saved / uploaded.
3. Judge select compiler and compile code from selected file.
4. Read the output and match it.
5. Return result and run scripts that add the result in a db.

To take the time for it to compile isn't nesiccary at the moment so i can figure on that one later. But so far I guess this works or I am totaly wrong.

Paying someone is out of the question like I said "No source code is wanted just some information on what I can/should read about to gain the knowledge to make one of my own." and if someone take charge for helping I guess I just wait for someone more polite to answer.

[Oler1s]

Quote

But so far I guess this works or I am totaly wrong.

Overall, very roughly, you have the right idea, but you aren't approaching the details correctly.

I don't have the time at this moment to point out what you should be looking at to understand and implement the details, although I'll take the time within the next few hours to tell you where to look.

Quote

Paying someone is out of the question like I said "No source code is wanted just some information on what I can/should read about to gain the knowledge to make one of my own." and if someone take charge for helping I guess I just wait for someone more polite to answer.

I wasn't asking for financial payment. I have a link in my signature named "what have you tried". Could you please read that? Ideally, you would read the other two links in my signature as well.

The thing is, you do have obligations as a poster. Those obligations aren't financial, but they are your effort, attitude, etc. and other prerequisites as you would expect when asking someone for free help. I always ask what people have tried both to gauge whether it's worth helping and also to give better answers. Telling me what you have done and what you are thinking allows me to help you better. It's a win win situation.

If you agree with these obligations, then I don't think you meant to call me impolite. Rather, you probably meant to say that my asking for such information was unusual.

There's more to coding in software development. As you know, a significant part of time is just figuring out and thinking about ways to tackle the problem. I wanted to know what your thoughts were so far. I don't think my responses are rude, and I don't presume that people asking questions are lazy or anything else. I just ask for information, and factor your information into my response. You certainly want the best possible answer, don't you?

[Oler1s]

So, the workflow is as follows. Person goes to website and uploads source file of some language, for some problem. Judge system processes file to executable based on whatever the language and toolchain is. Example, C files get compiled. Python compiled to bytecode (optionally). Java has its compiler. And so on. Then the actual program is run, and the output captured. Finally, the output is scored and the result displayed back to the person. There's four components to this workflow.

1) The website interface.
2) The compilation to a runnable file.
3) Running the file.
4) Output and scoring.

The website interface is just generic web development. You have user accounts, a database of scores, file uploads, etc. There's nothing particularly judge specific in this component.

(2) and (3) are coupled. (2) is more straightforward. If something needs to be compiled, it is. Otherwise, there's a source file that's run appropriately (directly, python interpreter, ruby interpreter, etc.).

Running the file isn't so straightforward. It seems like you can just call the file, and redirect output to a file. That's true, if the program works and does what it is supposed to do. What do you do about malicious programs? What about just bad mistakes like an infinite loop? There's various approaches to sandboxing programs, and you'll want to look into this complex topic.

The final part is relatively straightforward. I think it's more tedious because once capture output, you'll need to deal with issues like whitespace, etc. You don't want to flag someone's correct output as false because they started off with an unnecessary whitespace. It would seem just a bit strict if you took issues with whitespace, etc.

Scoring should be straightforward.

You'll want to work on each component one by one. They aren't tightly coupled so you can deal with each part of the judge system one by one.

You want to make sure the system is both secure and robust. Running the program is trickiest. The website part shouldn't be an issue if you follow good development practices and take advantage of frameworks, etc.

There aren't any constraints on programming languages, but you may end up using two or three or more depending on how you approach it. The website component needs some server side language to handle the uploads and authentication, etc.