PxPlus User Forum

Twitter Twitter Twitter

Author Topic: Reading simple data from a TCP channel  (Read 737 times)

martinp

  • Silver Member
  • ***
  • Posts: 37
    • View Profile
Reading simple data from a TCP channel
« on: March 23, 2023, 01:38:57 PM »
Hello everyone, just a simple question.

I have an incoming TCP data stream I want to capture.  For example port 9004.

I've created a simple incoming server routine.  Not sure if I needed KEEPALIVE but

let TCP=unt; open (TCP,bsz=4096,err=ERR_PORT)"[tcp];9004;KEEPALIVE"
READ_NEXT:
read record (TCP,err=ERR_READ)A$
print "Got: ",A$
goto READ_NEXT


Similarly I ran a linux "nc capture command" capturing data from another port which has the same data being sent to it.


What I am noticing is occasionally the PVX program does something like this:

Almost like a terminator character is received on the port and it registers part of the data then it continues on the next read fine.  So I would have to concatenate the data.

The nc command however does not break apart any data.

I am just wondering if I need to change anything in this routine.  Thanks.

« Last Edit: March 23, 2023, 01:46:19 PM by martinp »

Mike King

  • Diamond Member
  • *****
  • Posts: 3811
  • Mike King
    • View Profile
    • BBSysco Consulting
Re: Reading simple data from a TCP channel
« Reply #1 on: March 23, 2023, 01:44:52 PM »
You have specified a blocking factor of 4K so every time you read the most you will get back is 4K regardless of how much has been received.

Your PRINT command will then insert a visible line break on the output -- the line feed is not in the data but simply a result of the PRINT statement not having a trailing comma..

Try:

Code: [Select]
  let TCP=unt; open (TCP,bsz=4096,err=ERR_PORT)"[tcp];9004;KEEPALIVE"
READ_NEXT:
  read record (TCP,err=ERR_READ)A$
  print A$,
  goto READ_NEXT

Mike King
President - BBSysco Consulting
eMail: mike.king@bbsysco.com

martinp

  • Silver Member
  • ***
  • Posts: 37
    • View Profile
Re: Reading simple data from a TCP channel
« Reply #2 on: March 23, 2023, 01:48:11 PM »
Thanks Mike but this data is coming in very slow like as you can see by the time stamp.  Every few seconds very small data chunks? 

martinp

  • Silver Member
  • ***
  • Posts: 37
    • View Profile
Re: Reading simple data from a TCP channel
« Reply #3 on: March 23, 2023, 01:51:37 PM »
Let me resend with this picture I had this code with my time stamp

0040 let TCP=unt; open (TCP,bsz=4096,err=ERR_PORT)"[tcp];9004;KEEPALIVE"
0050 read record (TCP,err=ERR_READ)A$
0060 let TIME$=dte(0:%DATTIM_MSK$)
0070 print TIME$+"   Got: ",A$


Mike King

  • Diamond Member
  • *****
  • Posts: 3811
  • Mike King
    • View Profile
    • BBSysco Consulting
Re: Reading simple data from a TCP channel
« Reply #4 on: March 23, 2023, 01:55:10 PM »
Yes but TCP is a streaming transmission so arrival times are never assured.  A transmission error somewhere along the stream can cause a delay/small packet at any time.

Also, if you receiving small intermittent packets I would suggest you add the option NODELAY to disable the Nagles algorithm on the connection, it is normally enabled on TCP. (See Nagles Algorithm at https://en.wikipedia.org/wiki/Nagle%27s_algorithm).  It also possible the sender may need to disable the algorithm to provide an even flow of data.

All that being said, did the example I posted with the trailing comma resolve what you seeing?
Mike King
President - BBSysco Consulting
eMail: mike.king@bbsysco.com

martinp

  • Silver Member
  • ***
  • Posts: 37
    • View Profile
Re: Reading simple data from a TCP channel
« Reply #5 on: March 23, 2023, 01:59:20 PM »
Let me try the NODELAY I was thinking about that as well.

I don't think the comma will work because that output you were seeing was from a separate echo to a text file which I was just logging.  I didnt want to post a bunch of messy code :)

Its clear its hitting a second READ

0100 let TEMP$="echo '"+TIME$+" "+A$+"' >> /tmp/ima2_out.txt"

Thanks for your rapid responses :)

martinp

  • Silver Member
  • ***
  • Posts: 37
    • View Profile
Re: Reading simple data from a TCP channel
« Reply #6 on: March 23, 2023, 02:11:48 PM »
Just thinking about this again.  Are you saying I should always concatenate the receiving data from [tcp] and then check for a starting/termination string in order to qualify the received data.  I see.  I was just surprised the nc data was just perfectly clean so I was not sure.

Not used to reading data directly from TCP streams so this is very helpful. 

Mike King

  • Diamond Member
  • *****
  • Posts: 3811
  • Mike King
    • View Profile
    • BBSysco Consulting
Re: Reading simple data from a TCP channel
« Reply #7 on: March 23, 2023, 02:44:00 PM »
Yes -- since its a streaming interface and since TCP/IP itself breaks up transmissions then reassembles them when they are received there is never a 100% clean way to assure you got all the data unless you issue something like a READ RECORD (nn, SIZ=xx) R$ where you know you are expecting fixed size data chunks.

BTW: If you notice on the last PNG you posted the time stamp for the split transmission was identical.  Basically something in the TCP/IP protocol simply caused the blocks of data to arrive independently thus part of the transmission was received and passed to your program with the remainder arriving probably milliseconds later. 

This streaming of TCP data is generally why TCP transmissions normally are designed to provide length information (e.g. the Content-Length header on web requests) or have a signal (e.g. the empty line between web page headers and contents) that can be used to detected end of transmission.
Mike King
President - BBSysco Consulting
eMail: mike.king@bbsysco.com

martinp

  • Silver Member
  • ***
  • Posts: 37
    • View Profile
Re: Reading simple data from a TCP channel
« Reply #8 on: March 23, 2023, 03:37:34 PM »
Thank you Mike this is great.  Much appreciated!

In this case these are scanners.  What's neat is I can set the scanner to send the length of the data as part of the data stream, so I can use that as well to parse if needed.