ghqwerty's Profile User Rating: -----

Reputation: 43 Craftsman
Group:
Contributor w/DIC++
Active Posts:
903 (0.4 per day)
Joined:
08-August 08
Profile Views:
7,435
Last Active:
User is offline Aug 10 2014 03:48 PM
Currently:
Offline

Previous Fields

Country:
GB
OS Preference:
Windows
Favorite Browser:
FireFox
Favorite Processor:
Intel
Favorite Gaming Platform:
XBox
Your Car:
Who Cares
Dream Kudos:
100

Latest Visitors

Icon   ghqwerty has not set their status

Posts I've Made

  1. In Topic: Limit to fprintf or .dat file?

    Posted 10 Aug 2014

    Forgot the close file... Added it in and now it works properly. Thanks!
  2. In Topic: Limit to fprintf or .dat file?

    Posted 10 Aug 2014

    I will try to change the code to use c++ file streams. The reason I went with C files is because I find the modifier flags in printf much easier to use.

    The get averages function is:
    map_int_flt_vec CTicker::getAverages()
    {
    	if (movingAverages.size() >= 1)
    	{
    
    	}
    	else
    	{
    		calculateAverages();
    	}
    
    	return movingAverages;
    }
    



    returns a map of vectors where movingAverages["10"] is a vector for the 10 point moving average. movingAverages["20"] is for 20 point etc

    I have tested this too and it works fine
  3. In Topic: Is it quicker to read from a database or perform the calculations

    Posted 18 Jul 2014

    What do you mean, by the database? I wasn't aware they had such functionality?
  4. In Topic: Is it quicker to read from a database or perform the calculations

    Posted 18 Jul 2014

    Hi Jim, thanks for the response.

    Quote

    If they involve the entire data set then it would probably be better to store the raw data.

    This is pretty much the confirmation I was looking for. I was primarily wondering if the performance benefit of only having to calculate the data once would outweigh the effort required to maintain the database as it evolved. I get that this problem relies a hell of a lot on the implementation and the decision would depend on the requirements of the program and so either solution could be the best one.

    However, on a more fundamental level, how efficient is reading and writing from a database compared to performing operations in code? surely there a lot of overheads involved in having to open the db, find the correct rows and then return them to my program (the sqllite library is 100k+ lines, so surely that means that theres a lot going on?)
    Furthermore, when reading and writing from a database how does the complexity vary as the number of columns increase? Would it be linear or less than?

    E.g. would 8 columns take twice as long to read/write as 4?



    Quote

    No one here can possibly answer this question with the limited information you provided. This is a design decision that someone knowledgeable about the complete system should make. Perhaps you should discuss this matter with your superiors, colleagues, or the potential users of your system.


    This is why I put quotes around right. I now see this is more of a design decision than one based on the performance of the database. By the way, this is a personal project.


    Quote

    What does the raw data look like, and what calculations are to be done on that data?


    This is an example of a line of raw data.
    52 2014-07-14 95.70 96.06 95.54 95.84 19579000.00

    calculations include moving averages, highest and lowest value over a time period, the change in values from one day to the next.
    Eventually I also want to be able to 'trend spot'
  5. In Topic: Is it quicker to read from a database or perform the calculations

    Posted 18 Jul 2014

    I don't fully understand your response sorry, what do you mean by rehydrate the object?
    I should make clear that the calculations are being made in my C++ program, not the DB.

    It's a SQL database. I'll try better to demonstrate my question

    Read from files
    Perform calculations (can be more complex than just averages and sums) in c++ program
    Store ALL data

    Then when I next want to look at the data
    Read from db - This will give me every value straight away, however, every time I want to add a new value to analyse I will need to add a new column in the database and retrospectively fill it with data for all currently stored entries


    OR
    Read from files
    store raw data

    Then
    Read from DB
    Perform calculations in program

My Information

Member Title:
if($spareTime > 0){ $this->writeCode(); }
Age:
Age Unknown
Birthday:
Birthday Unknown
Gender:
Location:
study room
Years Programming:
2
Programming Languages:
php, css, html
learning - c#, javascript, jquery(javascript), ajax

Contact Information

E-mail:
Click here to e-mail me

Friends

Comments

ghqwerty has no profile comments yet. Why not say hello?