2 Replies - 909 Views - Last Post: 16 November 2011 - 02:57 PM Rate Topic: -----

#1 LivingNightmare   User is offline

  • D.I.C Head
  • member icon

Reputation: 21
  • View blog
  • Posts: 129
  • Joined: 07-July 10

C++ question about existing code

Posted 16 November 2011 - 12:15 PM

Hi everyone,

So I'm looking at an open source project called Piccolo on google code (http://code.google.com/p/piccolo/source/browse/src) and I have a question about something weird that I've found. So in the /src/master/master.cc file, there's a function in a struct as follows:

  // Order pending tasks by our guess of how large they are
  bool get_next(const RunDescriptor& r, KernelRequest* msg) {
    vector<TaskState*> p = pending();

    if (p.empty()) {
      return false;
    }

    TaskState* best = *max_element(p.begin(), p.end(), &TaskState::WeightCompare);

    msg->set_kernel(r.kernel);
    msg->set_method(r.method);
    msg->set_table(r.table->id());
    msg->set_shard(best->id.shard);

    best->status = TaskState::ACTIVE;
    last_task_start = Now();

    return true;
  }
};



Now, I can't find where the KernelRequest type was defined, so I searched the project on google code, and it looks like it's defined in this weird file src/worker/worker.proto - The file is a bit long, but I'm showing the relevant piece.
package dsm;

import "util/common.proto";
option optimize_for = SPEED;

enum CheckpointType {
  CP_NONE = 0;
  CP_MASTER_CONTROLLED = 1;
  CP_ROLLING = 2;
};

message RegisterWorkerRequest {
  required int32 id = 1;
}

message ShardAssignment {
  required int32 table = 1;
  required int32 shard = 2;
//  required int32 old_worker = 3;
  required int32 new_worker = 4;
}

message ShardAssignmentRequest {
  repeated ShardAssignment assign = 1;
}

message ShardInfo {
  required uint32 table = 1;
  required uint32 shard = 2;
  required uint64 entries = 3;
  required int32 owner = 4 [default = -1];
}

message MethodStats {
  required double total_time = 1;
  required double shard_time = 2;
  required int32 calls = 3;
  required int32 shard_calls = 4;
};

// !!!!!!!!!!!!! HERE !!!!!!!!!!!!!!
message KernelRequest {
  required string kernel = 1;
  required string method = 2;
  optional int32 table = 3;
  optional int32 shard = 4;
  required Args args = 5;
}

message KernelDone {
  required KernelRequest kernel = 1;

  // updated information about the state of this workers
  // table shards.
  repeated ShardInfo shards = 5;
}

...



Can anyone explain how this works? How is this a type... and and is it using the set_blah() methods without error? The second file doesn't really look like C++, but I have no idea how this works :\ - What kind of magic is this xD ?

L.N

This post has been edited by LivingNightmare: 16 November 2011 - 01:22 PM


Is This A Good Question/Topic? 0
  • +

Replies To: C++ question about existing code

#2 Aphex19   User is offline

  • Born again Pastafarian.
  • member icon

Reputation: 619
  • View blog
  • Posts: 1,873
  • Joined: 02-August 09

Re: C++ question about existing code

Posted 16 November 2011 - 02:45 PM

It seems that the .proto files implement something called Protocol buffers, but I don't know anymore than that.

Apparently, Protocol buffers are...

Quote

a language-neutral, platform-neutral, extensible way of serializing structured data for use in communications protocols, data storage, and more.

Was This Post Helpful? 1
  • +
  • -

#3 LivingNightmare   User is offline

  • D.I.C Head
  • member icon

Reputation: 21
  • View blog
  • Posts: 129
  • Joined: 07-July 10

Re: C++ question about existing code

Posted 16 November 2011 - 02:57 PM

Thanks :) - I think that this is enough information for what I'll need to do. The link you provided gives a pretty good explanation :)

L.N
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1