6 Replies - 2416 Views - Last Post: 30 December 2015 - 09:35 AM Rate Topic: -----

#1 kodabear  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 11
  • Joined: 28-December 15

`parse': Fatal error: Start tag expected, '<' not foun

Posted 28 December 2015 - 02:11 PM

For a few days now I have been trying to run this Sinatra program but i keep getting a `parse': Fatal error: Start tag expected, '<' not found at :1. (LibXML::XML::Error) when i try to create Hash of the XML data that the game i play give out. any advice of what i should do would be great help

[ruby - illyriad.rhcloud.com IllyriadAp]\ > ruby data_syndicator.rb
Creating data tables...
Feeds
Towns
Town Deltas
Races
Players
Alliances
Alliance Roles
  === PARSING TOWNDATA FEED ===
  Fatal error: Start tag expected, '<'
not found at: 1. /
  var / lib / openshift / 567a568c0c1e66874e000160 / .gem / gems / xml - object - 0.9.93 / lib / xml - object / adapters / libxml.rb: 9: in `parse': Fatal error: Start tag expected, '<' not found at :1. (LibXML::XML::Error)
        from /var/lib/openshift/567a568c0c1e66874e000160/.gem/gems/xml-object-0.9.93/lib/xml-object/adapters/libxml.rb:9:in `
new '
        from /var/lib/openshift/567a568c0c1e66874e000160/.gem/gems/xml-object-0.9.93/lib/xml-object.rb:15:in `new'
from /
  var / lib / openshift / 567a568c0c1e66874e000160 / ruby / IllyriadAp / data_syndicator / core.rb: 68: in `parseTownData'
        from /var/lib/openshift/567a568c0c1e66874e000160/ruby/IllyriadAp/data_syndicator/core.rb:54:in `
block(2 levels) in run!'
        from /opt/rh/ruby193/root/usr/share/ruby/open-uri.rb:150:in `open_uri'
from / opt / rh / ruby193 / root / usr / share / ruby / open - uri.rb: 677: in `open'
        from /opt/rh/ruby193/root/usr/share/ruby/open-uri.rb:33:in `
open '
        from /var/lib/openshift/567a568c0c1e66874e000160/ruby/IllyriadAp/data_syndicator/core.rb:53:in `block in run!'
from /
  var / lib / openshift / 567a568c0c1e66874e000160 / ruby / IllyriadAp / data_syndicator / core.rb: 50: in `each'
        from /var/lib/openshift/567a568c0c1e66874e000160/ruby/IllyriadAp/data_syndicator/core.rb:50:in `
run!'
        from data_syndicator.rb:17:in `<main>'



require 'sequel'
require 'net/http'
require 'open-uri'
require 'xml-object'
begin
    require 'xml-object/adapters/libxml'
rescue LoadError
    puts "LibXML not present, continuing with REXML..."
end


module IllyriadApi
    class DataSyndicator
    private
        @db = nil

    public
        def initialize(db = Sequel.sqlite)
            @db = db
        end

        def run!
            # Re-create tables
            puts "Creating data tables..."
            createTables!

            # Hash of all Illyriad data feeds
            feeds = {
                "towns" => {
                    :action => "parseTownData",
                    :url => "http://data-root.illyriad.co.uk/datafile_towns.xml" },
                "players" => {
                    :action => "parsePlayerData",
                    :url => "http://data-root.illyriad.co.uk/datafile_players.xml" },
                "alliances" => {
                    :action => "parseAllianceData",
                    :url => "http://data-root.illyriad.co.uk/datafile_alliances.xml" }
            }

            # Parse and syndicate each data feed
            feeds.each { |name, hash|
                puts "=== PARSING #{name.upcase} FEED ==="
                start = Time.now
                open(hash[:url]) { |feed|
                    send(hash[:action], feed)
                }
                duration = Time.now - start
                puts "Feed syndication completed in #{duration.to_s} seconds."
                puts
            }

            # Close DB connection
            puts "Disconnecting"
            @db.disconnect
        end

        def parseTownData(xml)
            start = Time.now
            townData = XMLObject.new xml
            duration = Time.now - start

            @db[:feeds].insert(
                :generated_at => townData.server.datagenerationdatetime,
                :type => "Towns",
                :is_current => true)

            puts "XML parsed in #{duration.to_s} seconds."
            puts "Towns found: #{townData.towns.count}"

            start = Time.now
            townData.towns.each { |town|
                @db[:towns].insert(
                    :data_timestamp => townData.server.datagenerationdatetime,
                    :town_id => town.towndata.townname[:id],
                    :name => town.towndata.townname,
                    :location_x => town.location.mapx,
                    :location_y => town.location.mapy,
                    :founded_at => town.towndata.foundeddatetime,
                    :owner_id => town.player.playername[:id],
                    :population => town.towndata.population,
                    :is_capital => town.towndata.iscapitalcity,
                    :is_alliance_capital => town.towndata.isalliancecapitalcity)
            }
            duration = Time.now - start
            puts "Database populated in #{duration.to_s} seconds."

            puts "Logging town deltas..."
            start = Time.now
            feedTimestamp = DateTime.parse(townData.server.datagenerationdatetime)
            logTownDeltas!(feedTimestamp)
            duration = Time.now - start
            puts "Town deltas logged in #{duration.to_s} seconds."
        end

        def logTownDeltas!(newDate)
            deltas = []

            oldTowns = @db[:towns].filter { data_timestamp < newDate }
            currentTowns = @db[:towns].except(oldTowns)

            destroyedTownIDs = oldTowns.select(:town_id).except(currentTowns.select(:town_id)).collect { |d| d[:town_id] }
            createdTownIDs = currentTowns.select(:town_id).except(oldTowns.select(:town_id)).collect { |c| c[:town_id] }

            alteredTowns = Hash.new
            currentTowns.each { |town|

            }

            destroyedTownIDs.each { |d|
                t = oldTowns.filter(:town_id => d).first
                @db[:town_deltas].insert(
                    :happened_at => newDate,
                    :town_id => d,
                    :owner_id => t[:owner_id],
                    :name => t[:name],
                    :population => 0,
                    :is_capital => 0,
                    :is_alliance_capital => 0)
            }

            createdTownIDs.each { |c|
                t = currentTowns.filter(:town_id => c).first
                @db[:town_deltas].insert(
                    :happened_at => newDate,
                    :town_id => c,
                    :owner_id => t[:owner_id],
                    :name => t[:name],
                    :population => t[:population],
                    :is_capital => t[:is_capital],
                    :is_alliance_capital => t[:is_alliance_capital])
            }

            # Delete old data, it's not needed anymore
            @db[:towns].filter{ data_timestamp < newDate }.delete
        end

        def parsePlayerData(xml)
            start = Time.now
            playerData = XMLObject.new xml
            duration = Time.now - start

            puts "XML parsed in #{duration.to_s} seconds."
            puts "Players found: #{playerData.players.count}"

            @db[:feeds].insert(
                :generated_at => playerData.server.datagenerationdatetime,
                :type => "Players",
                :is_current => true)

            start = Time.now
            playerData.players.each { |player|
                alliance_id = (player.allianceid[:id] == "0" ? nil : player.allianceid[:id])
                alliance_role_id = (alliance_id.nil? ? nil : player.allianceroleid[:id])

                @db[:players].insert(
                    :id => player.playername[:id],
                    :name => player.playername,
                    :race_id => player.race[:id],
                    :alliance_id => alliance_id,
                    :alliance_role_id => alliance_role_id)
            }
            duration = Time.now - start
            puts "Database populated in #{duration.to_s} seconds."
        end

        def parseAllianceData(xml)
            start = Time.now
            allianceData = XMLObject.new xml
            duration = Time.now - start

            puts "XML parsed in #{duration.to_s} seconds."
            puts "Alliances found: #{allianceData.alliances.count}"

            @db[:feeds].insert(
                :generated_at => allianceData.server.datagenerationdatetime,
                :type => "Alliance",
                :is_current => true)

            start = Time.now
            allianceData.alliances.each { |alliance|
                capital_last_moved_at = (alliance.alliancecapitallastmoved rescue nil)
                taxrate_last_changed_at = (alliance.alliancetaxratelastchanged rescue nil)
                @db[:alliance].insert(
                    :id => alliance.alliance[:id],
                    :ticker => alliance.allianceticker,
                    :name => alliance.alliance,
                    :founded_at => alliance.foundeddatetime,
                    :founded_by_player_id => alliance.foundedbyplayerid[:id],
                    :capital_town_id => alliance.alliancecapitaltownid[:id],
                    :member_count => alliance.membercount,
                    :total_population => (alliance.totalpopulation rescue 0),
                    :tax_rate => (alliance.alliancetaxrate.to_i) / 100.0,
                    :tax_rate_last_changed_at => taxrate_last_changed_at,
                    :capital_town_last_moved_at => capital_last_moved_at)

                alliance.roles.each { |role|
                    @db[:alliance_roles].insert(
                        :id => role.role[:id],
                        :name => role.role,
                        :alliance_id => alliance.alliance[:id],
                        :hierarchy_id => role.heirarchy[:id])
                }
            }
            duration = Time.now - start
            puts "Database populated in #{duration.to_s} seconds."
        end

        def createTables!
            if @db.table_exists? :feeds
                puts "Feeds (*already created)"
                @db[:feeds].update(:is_current => false)
            else
                puts "Feeds"
                @db.create_table :feeds do
                    primary_key :id
                    DateTime :generated_at, :index => true
                    String :type, :index => true
                    TrueClass :is_current
                end
            end

            if @db.table_exists? :towns
                puts "Towns (*already created)"
            else
                puts "Towns"
                @db.create_table :towns do
                    primary_key :id
                    DateTime :data_timestamp
                    Integer :town_id, :index => true
                    String :name, :index => true
                    Integer :location_x
                    Integer :location_y
                    DateTime :founded_at
                    Integer :owner_id, :index => true
                    Integer :population
                    TrueClass :is_capital
                    TrueClass :is_alliance_capital
                end
            end

            if @db.table_exists? :town_deltas
                puts "Town Deltas (*already created)"
            else
                puts "Town Deltas"
                @db.create_table :town_deltas do
                    primary_key :id
                    DateTime :happened_at
                    Integer :town_id, :index => true
                    Integer :owner_id, :index => true
                    String :name, :index => true
                    Integer :population
                    TrueClass :is_capital
                    TrueClass :is_alliance_capital
                end
            end

            if @db.table_exists? :races
                puts "Races (*already created)" 
            else
                puts "Races"
                @db.create_table :races do
                    Integer :id, :primary_key => true
                    String :name, :index => true
                end

                @db[:races].multi_insert([
                    {:id => 1, :name => "Human"},
                    {:id => 2, :name => "Elf"},
                    {:id => 3, :name => "Dwarf"},
                    {:id => 4, :name => "Orc"}])
            end

            puts "Players"
            @db.create_table! :players do
                Integer :id, :primary_key => true
                foreign_key :race_id, :races
                Integer :alliance_id, :null => true
                Integer :alliance_role_id, :null => true
                String :name, :index => true
            end

               puts "Alliances"
            @db.create_table! :alliances do
                Integer :id, :primary_key => true
                String :ticker, :index => true
                String :name, :index => true
                DateTime :founded_at
                Integer :founded_by_player_id
                Integer :capital_town_id
                Integer :member_count
                Integer :total_population
                BigDecimal :tax_rate, :size => [5, 3]
                DateTime :tax_rate_last_changed_at, :null => true
                DateTime :capital_town_last_moved_at, :null => true
            end

            puts "Alliance Roles"
            @db.create_table! :alliance_roles do
                Integer :id, :primary_key => true
                String :name, :index => true
                foreign_key :alliance_id, :alliances
                Integer :hierarchy_id
            end



            #addForeignKeys!
        end

        private
        def addForeignKeys!
            @db.alter_table(:alliances) do
                add_foreign_key [:founded_by_player_id], :players
                add_foreign_key [:capital_town_id], :towns
            end

            @db.alter_table(:players) do
                add_foreign_key [:alliance_id], :alliances
                add_foreign_key [:alliance_role_id], :alliance_roles
            end

            @db.alter_table(:towns) do
                add_foreign_key [:owner_id], :players
            end
        end
    end
end



Is This A Good Question/Topic? 0
  • +

Replies To: `parse': Fatal error: Start tag expected, '<' not foun

#2 xclite  Icon User is offline

  • I wrote you an code
  • member icon


Reputation: 1237
  • View blog
  • Posts: 4,028
  • Joined: 12-May 09

Re: `parse': Fatal error: Start tag expected, '<' not foun

Posted 28 December 2015 - 02:23 PM

It looks like it's failing to parse the XML from your link. Have you tried printing the data you received from the URL before creating the XMLObject?
Was This Post Helpful? 0
  • +
  • -

#3 kodabear  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 11
  • Joined: 28-December 15

Re: `parse': Fatal error: Start tag expected, '<' not foun

Posted 28 December 2015 - 03:13 PM

when i use i get alot of weird stuff. not sure if i am using the wrong printing code
Net::HTTP.get_print URI.parse('http://data-root.illyriad.co.uk/datafile_towns.xml')


L▒?B▒1b▒▒▒▒▒6{▒.▒ś▒▒▒j▒Q2K▒▒▒x+▒▒i▒▒▒]▒/▒▒▒▒▒▒(s[)>]Dӗ▒P▒▒E▒▒▒ֆ▒0dd▒<▒▒FsEm▒X!▒\$▒4զ▒▒lO▒▒|;t▒▒▒▒4▒▒X▒J▒▒▒▒^▒▒▒ȖTǧ▒]▒}v▒ˣXي▒▒qED▒TR▒▒▒D▒▒~▒▒▒^ܼ▒▒?*▒0Y▒▒▒a▒▒"▒▒yES#▒▒▒}▒▒Z▒▒xs▒▒r▒▒D4▒▒.▒C▒o▒▒▒▒CZ▒▒d▒▒▒▒3ӎ▒4▒k▒PZ▒$▒4▒▒I▒▒!▒�2˃XZ▒▒▒▒▒k▒u▒3(Rf▒G֩▒v▒xq▒▒Fz▒▒p(b[▒▒BNv7ۡAj_▒]▒▒▒;`▒1▒▒▒▒G▒▒R*▒▒ţ߮▒ַ▒G▒|▒▒▒▒▒"w|▒t▒_▒▒W▒SYt▒▒▒▒▒▒6▒▒7▒O▒"`▒▒

sry for the really bad english i mean when i try to print i get alot of weird stuff.
Was This Post Helpful? 0
  • +
  • -

#4 xclite  Icon User is offline

  • I wrote you an code
  • member icon


Reputation: 1237
  • View blog
  • Posts: 4,028
  • Joined: 12-May 09

Re: `parse': Fatal error: Start tag expected, '<' not foun

Posted 28 December 2015 - 03:22 PM

So this clearly isn't XML. You need to get what you're passing into XMLObject.new to actually be XML. What is the return type of open-uri's open(feed)?
Was This Post Helpful? 0
  • +
  • -

#5 kodabear  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 11
  • Joined: 28-December 15

Re: `parse': Fatal error: Start tag expected, '<' not foun

Posted 28 December 2015 - 04:29 PM

here is a link to the github project


I am somewhat new to Ruby apps/ Sinatra framework. I found this code on guthub and ones i asked the owner if i can use it. i started testing it and been learning alot.
Was This Post Helpful? 0
  • +
  • -

#6 xclite  Icon User is offline

  • I wrote you an code
  • member icon


Reputation: 1237
  • View blog
  • Posts: 4,028
  • Joined: 12-May 09

Re: `parse': Fatal error: Start tag expected, '<' not foun

Posted 28 December 2015 - 07:34 PM

I know the answer to the question I asked - the point of the question is to help you learn! You need to be able to dig in and answer these questions yourself. This is failing because the XMLObject can't parse what you're sending to it. To see why this doesn't work, you need to figure out what you're sending to it.
Was This Post Helpful? 0
  • +
  • -

#7 kodabear  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 11
  • Joined: 28-December 15

Re: `parse': Fatal error: Start tag expected, '<' not foun

Posted 30 December 2015 - 09:35 AM

I fixed the problem by creating a cron job to download the xml files everyday after the data has been updated on my server and used those for the Parse parts. Ones I find out away to fix this server unexpectedly closed network connection error I will be able to see if it really works
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1