1 Replies - 5723 Views - Last Post: 26 June 2013 - 06:19 AM

#1 q2q_expand  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 31
  • Joined: 05-May 13

SCOPE: ' ... does not implement inherited abstract member ...'

Posted 26 June 2013 - 05:35 AM

Heylo! :bigsmile:/>

I am using a language, which is part of the .NET framework - SCOPE: Structured Computation Optimized for Parallel Execution.

The language is basically a combination of SQL and C# code (easy to learn). The problem I am solving is as follows:

Quote

- Read a list of queries from a dictionary-file
- Compute basic statistics: mean/median/mode of #words in query
- Compute the powerset for each query (ref)
- Compute basic statistics: mean/median/mode of size of powerset for each query
- Look up each term in the powerset in the dictionary -> compute fraction of entries which can be looked up
- Compute basic statistics: mean/median/mode of fraction


The code for it is as follows:

Define parameters (input file: txt dictionary, output file: txt files with stats
//     INPUT FILE PATHS    
#DECLARE ScoreCandidate string = "asdf.txt"; 
#DECLARE ScoreThreshold string = "assdf.txt";


// OUTPUT FILE PATHS
#DECLARE Mode_WordCount string = "asdf.txt";
#DECLARE Mean_WordCount string = "qwer.txt";
#DECLARE Median_WordCount string = "awer.txt";

#DECLARE Mode_TokenCount string = "qtr.txt";
#DECLARE Mean_TokenCount string = "rttru.txt";
#DECLARE Median_TokenCount string = "rcbcb.txt";

#DECLARE Mode_TokenPercentage string = "fgh.txt";
#DECLARE Mean_TokenPercentage string = "qwerqwr.txt";
#DECLARE Median_TokenPercentage string = "qwer.txt";




Extract data from input file according to certain schema and filter it
// Read query-by-query
Data = EXTRACT Query:string,
               Suggestion:string,
               QueryMarket:string,
               SuggestionMarket:string,
               Score:double,
               SourceID
        FROM @ScoreCandidate
        USING DefaultTextExtractor();

// Only pick unique queries (discard duplicates, triplets, etc.)
UniqueQueries = SELECT DISTINCT Query
                FROM Data;




Compute the first important piece of information: wordCount, TokenCount, List of all tokens (i.e. powerset)

// Store wordCount, tokenCount and list of tokens for each query
UniqueQueriesWordToken = PROCESS UniqueQueries
                         PRODUCE Query,
                                 WordCount,
				 TokenCount,
				 TokenList
                         USING   WordTokenProcessor;


#CS
public class WordTokenProcessor : Processor
{
    public override Schema Produces(string[] requestedColumns, string[] args, Schema inputSchema)
    {
        return new Schema("Query:string, WordCount:int16, TokenCount:int32, TokenList:List<string>");
//          return new Schema("Query:string, WordCount:int16, TokenCount:int32");
    }
      
    public override IEnumerable<Row> UniqueQueriesWordTokenProc(RowSet inputRow, Row outputRow, string[] args)
    {
        foreach(Row row in inputRow.Rows)
        {

            string query = row[0].String;

		    // wordCount	
            int numWords = query.Split(' ').Length;


		    // tokenCount: n! * sum(1/k!, k, 0, n)
		    int numTokens = 0;
		    for (int i = 0; i <= numWords; i++)
		    {
	    		numTokens++;
			    numTokens = numTokens*i;
		    }
		    numTokens--;


		    // tokenList - create powerset of all strings (excluding string = query itself)
            QueryPowerSet pset = new QueryPowerSet();
            pset.MakeCharArray(query);
            pset.allNonEmptyPowersetPermutationsButQuery();
            List<string> pS = pset.PowerSet;


		    // ProcessorOutput
            outputRow[0] = query;
            outputRow[1] = numWords;
            outputRow[2] = numTokens;
		    outputRow[3] = TokenList;
    
            yield return outputRow;
                         
       }

    }

}


public class QueryPowerSet
{

    private string[] inputSet;
    private int numberOfElements;
    private List<string> powerSet;
    public List<string> PowerSet
    {
        get
        {
            return powerSet;
        }
    }



    public void MakeCharArray(string InputString)
    {
        inputSet = InputString.Split(' ');
        numberOfElements = inputSet.Length;
    }



    public bool next_permutation(int[] perm, int n)
    {

        int k = -1;
        for (int i = 1; i < n; ++i)
        if (perm[i - 1] < perm[i])
            k = i - 1;
        if (k == -1)
        {
            Array.Reverse(perm, 0, n);
            return false;
        }

        int l = k + 1;
        for (int i = l; i < n; ++i)
            if (perm[k] < perm[i])
                    l = i;

        int t = perm[k];
        perm[k] = perm[l];
        perm[l] = t;
        Array.Reverse(perm, k + 1, n - k - 1);

        return true;

    }



    public void allNonEmptyPowersetPermutationsButQuery()
    {
        int[] v = new int[numberOfElements];
    	int[] idx = new int[numberOfElements];
        powerSet = new List<string>();

		for (int arraySize = 1; arraySize <= numberOfElements; arraySize++)
		{

			v[numberOfElements - arraySize] = 1;

            do
            {

                int cnt = 0;
                for (int i = 0; i < numberOfElements; ++i)
                {
                    if (v[i] != 0)
                        idx[cnt++] = i;
                }

                for (bool cond = arraySize < numberOfElements || next_permutation(idx, cnt); cond; cond = next_permutation(idx, cnt))
                {
                    string element = "";
                    for (int i = 0; i < cnt; ++i)
                    {
                        element = element + inputSet[idx[i]];
                        if (i + 1 < cnt)
                        {
                            element = element + " ";
                        }
                    }
                    powerSet.Add(element);
//                    Console.Write(element);
//                    Console.Write("\r\n");
                }

            } while (next_permutation(v, v.Length));

		}
    }


}
#ENDCS



Lookup the tokens in the original dictionary and compute fraction of tokens contained in dictionary

// Compute tokenContainedPercentage for each query
UniqueQueriesWordTokenPercentage = PROCESS UniqueQueriesWordToken
                              	   PRODUCE Query, 
                                           WordCount,
				                           TokenCount,
				                           TokenList,
					                       TokenPercentage
                              	   USING   TokenPercentageProcessor;

#CS
public class TokenPercentageProcessor : Processor
{
    public override Schema Produces(string[] requestedColumns, string[] args, Schema inputSchema)
    {
        return new Schema("Query:string, WordCount:int16, TokenCount:int32, TokenList:List<string>, TokenPercentage:double");
    }
      
    public override IEnumerable<Row> UniqueQueriesWordTokenPercentageProc(RowSet inputRow, Row outputRow, string[] args)
    {

        // write all queries into list
        List<string> QuerySet;
        foreach(Row row inputRow.Rows)
        {
            QuerySet.Add(row[0]);		                // load first column of all row-values into list
        }

        foreach(Row row in inputRow.Rows)
        {

            // count number of tokens contained in querySet
            List<double> ListOfTokens = row[4];
            int counter = 0;

            foreach(string element1 in QuerySet)
            {
   	            foreach(string element2 in ListOfTokens)
	            {
		            if (strcmp(element1, element2) == 0)
		            {
			            counter++;
		            }
	            }
            }

		    // Compute isContainedPercentage
		    int TokenCount = Convert.ToInt32(row[2]);
		    double TokenPercentage = counter/TokenCount.ConvertToDouble;


		    // ProcessorOutput
            outputRow[0] = row[0];
            outputRow[1] = row[1];
            outputRow[2] = row[2];
		    outputRow[3] = row[3];
		    outputRow[4] = TokenPercentage;

    
            yield return outputRow;
                         
        }
    }


    public static int mystrcmp(string st1, string st2)
    {
            int iST1 = 0, iST2=0;
            for (int i = 0; i < (st1.Length > st2.Length ? st1.Length : st2.Length); i++)
            {
                iST1 += (i >= st1.Length ? 0 : st1[i]) - (i >= st2.Length ? 0 : st2[i]);
                if (iST2 < 0)
                {
                    if (iST1 < 0)
                        iST2 += iST1;
                    if (iST1 > 0)
                        iST2 += -iST1;
                }
                else
                {
                    iST2 += iST1;
                }
            }
            return iST2;
     }


}
#ENDCS



Finally, we compute mean/median and mode for all three parameters

// Compute stats (mean, median, mode) for wordCount, tokenCount, tokenPercentage
// and write them into output file

// Mode
wordCount_Mode = SELECT DISTINCT Query,
                    		     WordCount,
                                 COUNT(*) AS Word_Freq 
    		     FROM UniqueQueriesWordTokenPercentage;

wordCount_Mode = REDUCE wordCount_Mode 
    		     ON Query 
    		     PRODUCE Mode 
    		     USING GetMode
    		     PRESORT Word_Freq ASC;

OUTPUT wordCount_Mode TO @Mode_WordCount;


tokenCount_Mode = SELECT DISTINCT Query,
                    		      TokenCount,
                                  COUNT(*) AS Token_Freq 
    		      FROM UniqueQueriesWordTokenPercentage;

tokenCount_Mode = REDUCE tokenCount_Mode 
    		      ON Query 
    		      PRODUCE Mode 
    		      USING GetMode
    		      PRESORT Token_Freq ASC;

OUTPUT tokenCount_Mode TO @Mode_TokenCount;


tokenPercentage_Mode = SELECT DISTINCT Query,
                    		           TokenPercentage,
                                       COUNT(*) AS Percentage_Freq 
    		           FROM UniqueQueriesWordTokenPercentage;

tokenPercentage_Mode = REDUCE tokenPercentage_Mode 
    		           ON Query
    		           PRODUCE Mode 
    		           USING GetMode
    		           PRESORT Percentage_Freq ASC;

OUTPUT tokenPercentage_Mode TO @Mode_TokenPercentage;


// Median
wordCount_Median = REDUCE UniqueQueriesWordTokenPercentage 
    		       ON Query 
    		       PRODUCE Median
    		       USING GetMedian
    		       PRESORT WordCount ASC;

OUTPUT wordCount_Median TO @Median_WordCount;

tokenCount_Median = REDUCE UniqueQueriesWordTokenPercentage 
    		        ON Query 
    		        PRODUCE Median
    		        USING GetMedian
    		        PRESORT TokenCount ASC;

OUTPUT tokenCount_Median TO @Median_TokenCount;

tokenPercentage_Median = REDUCE UniqueQueriesWordTokenPercentage 
    		   	         ON Query 
    		   	         PRODUCE Median
    		   	         USING GetMedian
    		   	         PRESORT TokenPercentage ASC;

OUTPUT tokenPercentage_Median TO @Median_TokenPercentage;


// Mean
wordCount_Mean = SELECT DISTINCT Query, 
                                 AVG(WordCount) AS Mean
    		     FROM UniqueQueriesWordTokenPercentage;

OUTPUT wordCount_Mean TO @Mean_WordCount;

tokenCount_Mean = SELECT DISTINCT Query, 
                                  AVG(TokenCount) AS Mean 
    		      FROM UniqueQueriesWordTokenPercentage;

OUTPUT tokenCount_Mean TO @Mean_TokenCount;

tokenPercentage_Mean = SELECT DISTINCT Query, 
                                       AVG(TokenPercentage) AS Mean
    		           FROM UniqueQueriesWordTokenPercentage;

OUTPUT tokenPercentage_Mean TO @Mean_TokenPercentage;



#CS
public class GetMode : Reducer
{
    public override Schema Produces(string[] requestedColumns, string[] args, Schema input)
    {
        return new Schema("Mode:double");
    }

    public override IEnumerable<Row> Reduce(RowSet input, Row outputRow, string[] args)
    {            
        string query = "";
        double mode = 0;
        foreach (Row r in input.Rows)
        {
            SourceID = r[0].String;
            mode = r[1].Double;
        }        
        outputRow[0].Set(mode);
        yield return outputRow;
    }
}

public class GetMedian : Reducer
{
    public override Schema Produces(string[] requestedColumns, string[] args, Schema input)
    {
        return new Schema("Median:double");
    }

    public override IEnumerable<Row> Reduce(RowSet input, Row outputRow, string[] args)
    {            
        string query = "";
        List<int32> list = new List<double>();
        foreach (Row r in input.Rows)
        {
            query = r[0].String;
            double cnt = r[1].Double;
            list.Add(cnt);  
        }
        double mid  = Convert.ToDouble(list.Count)/2;
        int medianindex =  Convert.ToInt32(Math.Ceiling(mid));
        
        outputRow[0].Set(list[medianindex]);
        yield return outputRow;
    }
}

#ENDCS




Now - when I compile this script, I get the following error:

Quote

Error compiling C# code: "'___Scope_Generated_Classes___.WordTokenProcessor.UniqueQueriesWordTokenProc(ScopeRuntime.RowSet, ScopeRuntime.Row, string[])': no suitable method found to override"
------
(9) using System.Runtime.Serialization;
(10) using System.Linq;
(11) public class ___Scope_Generated_Classes___
(12) {
(13)
(14) public class WordTokenProcessor : Processor
>>> CS0534: '___Scope_Generated_Classes___.WordTokenProcessor' does not implement inherited abstract member 'ScopeRuntime.Processor.Process(ScopeRuntime.RowSet, ScopeRuntime.Row, string[])'
(15) {
(16) public override Schema Produces(string[] requestedColumns, string[] args, Schema inputSchema)
(17) {
(18) return new Schema("Query:string, WordCount:int16, TokenCount:int32, TokenList:List<string>");
(19) // return new Schema("Query:string, WordCount:int16, TokenCount:int32");
(20) }
(21)
(22) public override IEnumerable<Row> UniqueQueriesWordTokenProc(RowSet inputRow, Row outputRow, string[] args)
>>> CS0115: '___Scope_Generated_Classes___.WordTokenProcessor.UniqueQueriesWordTokenProc(ScopeRuntime.RowSet, ScopeRuntime.Row, string[])': no suitable method found to override
(23) {
(24) foreach(Row row in inputRow.Rows)
(25) {
(26)
(27) string query = row[0].String;

[...]

(157) }
(158) }
(159)
(160)
(161) }
(162) public class TokenPercentageProcessor : Processor
>>> CS0534: '___Scope_Generated_Classes___.TokenPercentageProcessor' does not implement inherited abstract member 'ScopeRuntime.Processor.Process(ScopeRuntime.RowSet, ScopeRuntime.Row, string[])'
(163) {
(164) public override Schema Produces(string[] requestedColumns, string[] args, Schema inputSchema)
(165) {
(166) return new Schema("Query:string, WordCount:int16, TokenCount:int32, TokenList:List<string>, TokenPercentage:double");
(167) }
(168)
(169) public override IEnumerable<Row> UniqueQueriesWordTokenPercentageProc(RowSet inputRow, Row outputRow, string[] args)
>>> CS0115: '___Scope_Generated_Classes___.TokenPercentageProcessor.UniqueQueriesWordTokenPercentageProc(ScopeRuntime.RowSet, ScopeRuntime.Row, string[])': no suitable method found to override
(170) {
(171)
(172) // write all queries into list
(173) List<string> QuerySet;
(174) foreach(Row row inputRow.Rows)

**************
Stack Trace:
at ScopeCompiler.CSharpCodeCompiler.ThrowError(CompilerErrorCollection errors, String program)
at ScopeCompiler.CSharpCodeCompiler.Compile(String program, List`1 references, String assemblyName, CompileLogger logger)
at ScopeCompiler.CSharpCodeCompiler.CompileAndLoadAssembyImpl(String program, List`1 references, String assemblyName, CompileLogger logger, String key)
at ScopeCompiler.CSharpCodeCompiler.CompileAndLoadAssemby(String program, List`1 references, String assemblyName, CompileLogger logger)
at ScopeCompiler.CodeCompiler.Compile(String csharpProgram)
at ScopeCompiler.ScriptParser.HandleCSharpCode(Compiler compiler)
at ScopeCompiler.ParseSandboxer.FrontEndRun(String script, Compiler compiler, Settings settings)
at ScopeCompiler.ParseSandboxer.Parse(String script, Settings settings, CompilerEnvironment environment, CompilationContext context) C:\Users\t-ajgup\Documents\Visual Studio 2012\Projects\Testscripts\Training_Data_Extraction\Training_Data_Extraction\Tokenization.script



I don't see what the problem is with UniqueQueriesWordTokenProc? I am not overwriting any method here?

This post has been edited by q2q_expand: 26 June 2013 - 05:41 AM


Is This A Good Question/Topic? 0
  • +

Replies To: SCOPE: ' ... does not implement inherited abstract member ...'

#2 andrewsw  Icon User is online

  • It's just been revoked!
  • member icon

Reputation: 3615
  • View blog
  • Posts: 12,439
  • Joined: 12-December 12

Re: SCOPE: ' ... does not implement inherited abstract member ...'

Posted 26 June 2013 - 06:19 AM

I've only glance at your question, but isn't the message quite clear? And you are over-writing (overriding):

public override IEnumerable<Row> UniqueQueriesWordTokenProc(RowSet inputRow, Row outputRow, string[] args)

You are over-riding a method that either doesn't exist (doesn't have this name) or doesn't have the same signature (order and type of arguments).

If, as you description suggests, this is a completely new method, then remove the override modifier.

This post has been edited by andrewsw: 26 June 2013 - 06:22 AM

Was This Post Helpful? 1
  • +
  • -

Page 1 of 1