3 Replies - 963 Views - Last Post: 18 December 2014 - 06:01 AM Rate Topic: -----

#1 Lynchie435   User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 18
  • Joined: 08-December 14

Best way to communicate between two programmes

Posted 17 December 2014 - 10:28 AM

Afternoon,

I'm looking for some advice on a project I am working on.

Presently I have 2 Console Applications (That run 2 separate API's one for the phone system ACD and one of the Call Recorder).

These churn through traffic and store information in a database ready for a client application (Windows Form) to obtain and use.

At present I am storing the information in a SQL Table and using calls to and from the SQL database in both the Console Applications and Windows Form application.

Is this best practice? In my experience under load the SQL Server could slow down and I don't like the idea of relying on DB requests to send the information back.

Is there a better way I could communicate between my Console Applications and my WIndows Form? I was potentially thinking that TCP messages from one to the other on a port would work and then use the Windows Form to use the 'stream' to perform its actions rather than getting the information from the database.

The reason being is ideally I'd like to have the 'database' as stored within memory for quicker retrieval than using the SQL DB.

Any and all advice is fully appreciated. Please give me an explanation as to why your suggestion is efficient as I would like to learn from this rather than just 'do'

Many Thanks!

James

This post has been edited by andrewsw: 17 December 2014 - 12:38 PM
Reason for edit:: Removed (VB.NET) from the title


Is This A Good Question/Topic? 0
  • +

Replies To: Best way to communicate between two programmes

#2 modi123_1   User is online

  • Suitor #2
  • member icon



Reputation: 14417
  • View blog
  • Posts: 57,803
  • Joined: 12-June 08

Re: Best way to communicate between two programmes

Posted 17 December 2014 - 12:18 PM

Quote

At present I am storing the information in a SQL Table and using calls to and from the SQL database in both the Console Applications and Windows Form application.

Is this best practice?

Yes, that sounds fine.

Quote

In my experience under load the SQL Server could slow down and I don't like the idea of relying on DB requests to send the information back.

What database are you using? What sort of system is it? You can throw a whole mess of activity at a DB (assuming you are opening and closing connections right, good sql db calling practices, indexing, etc) without a large problem.

Quote

Is there a better way I could communicate between my Console Applications and my WIndows Form? I was potentially thinking that TCP messages from one to the other on a port would work and then use the Windows Form to use the 'stream' to perform its actions rather than getting the information from the database.

This seems like a bad idea. What happens if the winform is closed? What about logging, data review, etc that all come with having a DB in the middle?
Was This Post Helpful? 0
  • +
  • -

#3 thecoat   User is offline

  • D.I.C Addict

Reputation: 153
  • View blog
  • Posts: 537
  • Joined: 07-December 13

Re: Best way to communicate between two programmes

Posted 17 December 2014 - 11:27 PM

As modi123_1 already pointed out you aren't "doing it wrong", and using SQL as you are seems perfectly valid. What I will add is that in most cases when people have issues with SQL performance it's not really rooted in SQL but in the database or application design. Obviously I have no idea about your level of expertise with SQL, so if you are a SQL guru and know for certain that your database is absolutely optimized and there aren't any scale-up tweaks you could do then take that with a grain of salt.

The one draw back I see with SQL is that it sounds like you are running frequent queries on your client app, and yea tweaking the frequency and finding a balance between update speed and network load could be a royal pain. I couldn't think of an out of the box way to have SQL push the data to your app... but there is this:

http://technet.micro...(v=sql.10).aspx

I haven't played with StreamInsight or it's predecessor but it looks like it might offer you the capability of pushing events and data out of SQL server to your app.


I have created a pair of apps that talk to each other using TCP socket communications, one of them runs as a windows service and the other is a client app that can show various aspects of the service status and processing and send commands to it (not windows service start/stop/pause, those are built into .net libraries, these are specialized commands the running service sends out or reacts to). Although I have it in a production environment at multiple sites and have had very little problems with it I still question if this was the best approach. That being said there were some hurdles to over come and if you are leaning on doing it this way:

1.)Get cozy with threading - particularly on the client side app you are going to want to open a socket connection and respond to updates from the server without blocking your main thread where your user is trying to get work done. Any reactive gui updates are going to be called from something other than the main thread and will therefore have to be marshaled back to the main thread. Suddenly instead of going Textbox1.Text = "some value" you have to create a delegate and a method to set the text of the text box. The good news is you can somewhat make your methods generic enough that you can pass the form object to be updated and the value to update it with instead of multiple methods for the same control types using the same delegate.

On the server side you'll likely need to use threading to handle incoming and outgoing data and supporting multiple clients can get rather interesting, as you will need to keep track of them.

2.)TCP/IP socket clients can just disappear and there is no real good way to check before you transmit data that the socket is still connected. A client can gracefully close the socket at which point checking the status of the socket on the server will reveal the socket is disconnected, however if the client crashes or for some other reason (loss of power, user end tasks it etc) the client doesn't gracefully disconnect, the server side check will tell you that it's still connected. If you allow for multiple clients, this can mean over time that you have a bunch of clients that the server thinks are still there, but really aren't... ie you get a nice big memory leak. There are two approaches to dealing with this, one is to establish a connection on yet another socket where you constantly stream a heartbeat message from the client. I don't care for this because it assumes that if you have a heartbeat on the 2nd socket the first must be connected, which isn't necessarily true (although likely) and it also generates a lot of network traffic. What I did is in my main loop was to send a response challenge to each client and wait a configurable amount of time for a response, if the client fails to respond then you close the socket from the server side.

3.)You can break down your message into a binary stream and send it over the socket, but what I ended up doing was sharing a common class reference between the two applications, and making that class serializable. What's cool about that is you define your message as an object, and can have enumerations for the type of message. So my object might have an enumeration for object type "GUI Update", "Command", "KeepAlive". If it's a GUI update there may be a property for the type of GUI update and so forth so that you can create message processing code that follows a hierarchical chain of logic. Both sides expect the one object type on either end, so to send a message you create an instance of the object, serialize it and send it over the wire, on the other side you receive it, deserialize it and process it. You only need one socket and can always cast it to the appropriate object. Of course this means both side have to have the same version of the object library etc. This makes it fairly easy to add to the types of updates or commands and you don't start having to worry about buffer sizes and repeated reads to make sure you get all the data before converting and parsing the command. Of course that comes at a cost in that the update command does include default property values for the GUI Update properties, but I haven't really had an issue with socket communications getting burdened with large objects or the like... but who knows it could all come crashing down on my head at some point and I'll have to redo the entire thing ><.

4.)Do a lot of testing and do everything you can think of to break things... I did a lot of tweaking to resolve dumb little issues I found. Between threading and things that can crop up with IP communication there's a lot to account for and a lot that can go wrong that's hard to produce or reproduce... on that note it's also a good idea to at least be able to turn on very detailed logging.

5.)Lastly if you need to persist your data, don't completely drop sql. You could still do both, when you get data into the app, save it to sql as well as send it to the client app. That being said if you aren't pretty familiar with either threading or socket communications, this would probably be a nightmare to try and do.

I really just blathered on here to give you an idea of what you may be in for if you choose this route as it sounds good on the surface but the details can get gory really fast, I think you'll likely be far better off working to improve your existing design.
Was This Post Helpful? 0
  • +
  • -

#4 Lynchie435   User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 18
  • Joined: 08-December 14

Re: Best way to communicate between two programmes

Posted 18 December 2014 - 06:01 AM

Hi Both,

Thank you so much for your replies especially yours thecoat, very in-depth and made for great reading.

I think as my Phase 1 roll-out I am going to stick with SQL as its what I know. I can understand what you are saying about sockets and that is something I need to bear in mind.

Regards,

James
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1