Read file with escaped newlines into array

Read file with escaped newlines into array

[Sebastian Gniazdowski]

Hello
fc -W stores history to a given file. It escapes newlines with \. How
to read such file into array and have "\^M" lines put back together
into single array entries?

Best regards,
Sebastian Gniazdowski

Re: Read file with escaped newlines into array

[Bart Schaefer]

On Nov 19,  6:34pm, Sebastian Gniazdowski wrote:
}
} fc -W stores history to a given file. It escapes newlines with \. How
} to read such file into array and have "\^M" lines put back together
} into single array entries?

The best way is probably to use the history mechanism to read it:

    () { fc -ap -R the_given_file ; the_desired_array=( ${history} ) }

Although the docs don't yet say so, the special associative array
$history is not implemented as a hash table, so the entries are
guaranteed to be in a fixed order from most recent (largest history
number) to longest ago.  Which is the reverse of the way you'd read
them from the file, so you might want ${(Oa)the_desired_array} when
referencing.

Re: Read file with escaped newlines into array

[Sebastian Gniazdowski]

On 19 November 2015 at 20:17, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Nov 19,  6:34pm, Sebastian Gniazdowski wrote:
> }
> } fc -W stores history to a given file. It escapes newlines with \. How
> } to read such file into array and have "\^M" lines put back together
> } into single array entries?
>
> The best way is probably to use the history mechanism to read it:
>
>     () { fc -ap -R the_given_file ; the_desired_array=( ${history} ) }
>
> Although the docs don't yet say so, the special associative array
> $history is not implemented as a hash table, so the entries are
> guaranteed to be in a fixed order from most recent (largest history
> number) to longest ago.  Which is the reverse of the way you'd read
> them from the file, so you might want ${(Oa)the_desired_array} when
> referencing.

The way $history is implemented is crucial to me and solves my
problem. I needed ordered history started from most recent, without
"\n" substituted for newlines, and "$history[@]" does that. Thanks.
This could be mentioned in documentation just to protect that
functionality

Best regards,
Sebastian Gniazdowski

Re: Read file with escaped newlines into array

[Sebastian Gniazdowski]

On 19 November 2015 at 20:17, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Nov 19,  6:34pm, Sebastian Gniazdowski wrote:
> }
> } fc -W stores history to a given file. It escapes newlines with \. How
> } to read such file into array and have "\^M" lines put back together
> } into single array entries?
>
> The best way is probably to use the history mechanism to read it:
>
>     () { fc -ap -R the_given_file ; the_desired_array=( ${history} ) }

This works, but I wonder, how it's possible that zsh/parameter
variables such as $historywords are made local?

Best regards,
Sebastian Gniazdowski

Re: Read file with escaped newlines into array

[Bart Schaefer]

On Tue, May 10, 2016 at 7:26 AM, Sebastian Gniazdowski
<sgniazdowski@gmail.com> wrote:
> On 19 November 2015 at 20:17, Bart Schaefer <schaefer@brasslantern.com> wrote:
>>
>>     () { fc -ap -R the_given_file ; the_desired_array=( ${history} ) }
>
> This works, but I wonder, how it's possible that zsh/parameter
> variables such as $historywords are made local?

Special parameters can be made local simply by declaring them so.  Or
are you asking "how" in the sense of what is done internally to make
it work?

In the case of $history et al., internally they reference the history
data structures directly, so "fc -p" implicitly makes $history refer
to the new temporary history, and then switch back to the real history
on "fc -P"; it isn't necessary to declare it local when using "fc
-ap".

Or perhaps you're asking how to populate a local variable called
"history" that does NOT track other history changes?  In that case you
need to declare it with "local -h history" -- but if you do so, then
the real $history is hidden from all functions deeper in the call
stack, there is no way to get back to having $history refer to the
actual current history state until the declaring function returns.

Re: Read file with escaped newlines into array

[Sebastian Gniazdowski]

On 10 May 2016 at 18:15, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Tue, May 10, 2016 at 7:26 AM, Sebastian Gniazdowski
> <sgniazdowski@gmail.com> wrote:
>> On 19 November 2015 at 20:17, Bart Schaefer <schaefer@brasslantern.com> wrote:
>>>
>>>     () { fc -ap -R the_given_file ; the_desired_array=( ${history} ) }
>>
>> This works, but I wonder, how it's possible that zsh/parameter
>> variables such as $historywords are made local?
>
> Special parameters can be made local simply by declaring them so.  Or
> are you asking "how" in the sense of what is done internally to make
> it work?
>
> In the case of $history et al., internally they reference the history
> data structures directly, so "fc -p" implicitly makes $history refer
> to the new temporary history, and then switch back to the real history
> on "fc -P"; it isn't necessary to declare it local when using "fc
> -ap".

I use $history to refer to main history and also load a private
history by using the code you gave, and they don't interfere. You used
anonymous function suggesting $history is being localized and that's
how it in practice works. Read the manual (second time), it's -a that
makes the temporary structure switched to by fc -p to be automatically
dropped. Convenient.

Best regards,
Sebastian Gniazdowski

Re: Read file with escaped newlines into array

[Mikael Magnusson]

On Thu, Nov 19, 2015 at 6:34 PM, Sebastian Gniazdowski
<sgniazdowski@gmail.com> wrote:
> Hello
> fc -W stores history to a given file. It escapes newlines with \. How
> to read such file into array and have "\^M" lines put back together
> into single array entries?

Also note that the history file is stored metafied, so even if you
take care of escaped newlines, you still would need to unmetafy it.

-- 
Mikael Magnusson

unmetafy.c

#define Meta ((char) 0x83)

#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>

/* from zsh utils.c */
char *unmetafy(char *s, int *len)
{
  char *p, *t;

  for (p = s; *p && *p != Meta; p++);
  for (t = p; (*t = *p++);)
    if (*t++ == Meta)
      t[-1] = *p++ ^ 32;
  if (len)
    *len = t - s;
  return s;
}

int main(int argc, char *argv[]) {
  char *line = NULL;
  size_t size;

  while (getline(&line, &size, stdin) != -1) {
    unmetafy(line, NULL);
    printf("%s", line);
  }

  if (line) free(line);
  return EXIT_SUCCESS;
}