Problem with redirecting from serial device

Problem with redirecting from serial device

[Ian Petts]

I have been playing around with weather station receiver software on the
Arduino. The Arduino outputs serial data over USB on device ttyACM0.

The incoming data is not timestamped, so I have a small awk script to add this
as the data is received:

#!/usr/bin/awk -f

{
    dt = strftime("%Y-%m-%d %H:%M:%S", systime(), 0)
    print dt " " $0
}

When I run the script with output going to the terminal, everything looks to
be working just fine:

ian@nibbler[1568]~% ./dt-add.awk < /dev/ttyACM0
2013-07-18 15:34:18 [OS V3 Sensor Capture/Decode]
2013-07-18 15:35:16 Wind Gust: 2.33    Wind Avg: 2.42    Dir: NW
2013-07-18 15:36:02 Temp: 17.11   Humidity: 61
2013-07-18 15:36:55 Temp: 17.01   Humidity: 61

As soon as I add a redirect or pipe to the above (to save the data to a file
or for further processing) no data appears:

ian@nibbler[1570]~% ./dt-add.awk < /dev/ttyACM0 > weather.out
# A few minutes later
^C
ian@nibbler[1571]~% ls -lh weather.out
-rw-rw-r-- 1 ian ian 0 Jul 18 15:47 weather.out

I don't think it's a file buffering issue, I've left it go for several hours
with the same results. Even if I try 'more'ing the data I seem to get no
output.

Dumping the raw data to a file seems to be fine.
< /dev/ttyACM0 > foo
works as expected.

Can anyone please explain where the issue is?

-- 
All computers run at the same speed...with the power off.

Re: Problem with redirecting from serial device

[Phil Pennock]

On 2013-07-18 at 15:59 +1000, Ian Petts wrote:
> As soon as I add a redirect or pipe to the above (to save the data to a file
> or for further processing) no data appears:

> I don't think it's a file buffering issue, I've left it go for several hours
> with the same results. Even if I try 'more'ing the data I seem to get no
> output.

To rule it out, could you try gawk explicitly if awk is not gawk?

  http://www.perkin.org.uk/posts/how-to-fix-stdio-buffering.html

I hadn't heard of stdbuf before, but like the idea, using an LD_PRELOAD
hack to be able to fix what is a userland problem.

-Phil

Re: Problem with redirecting from serial device

[Ian Petts]

> > As soon as I add a redirect or pipe to the above (to save the data to a file
> > or for further processing) no data appears:
> 
> > I don't think it's a file buffering issue, I've left it go for several hours
> > with the same results. Even if I try 'more'ing the data I seem to get no
> > output.
> 
> To rule it out, could you try gawk explicitly if awk is not gawk?

Thanks for the info, Phil. My awk is indeed gawk and I also tried the stdbuf
command per the link you supplied. Unfortunately I am still seeing the same
results.

It is starting to look like a buffering problem, however, because using perl
with output buffering on (the default), does not appear to work
< /dev/ttyACM0 perl -MPOSIX -e '$| = 0;' -pe 'print strftime("%F %T ", localtime)' > weather.out

Turn output buffering off, data is written to the weather.out file
< /dev/ttyACM0 perl -MPOSIX -e '$| = 1;' -pe 'print strftime("%F %T ", localtime)' > weather.out

Either case above without the redirect displays the data to the screen.

-- 
Who is General Failure and why is he reading my disk?