[9fans] International Ispell in Plan9

[9fans] International Ispell in Plan9

[trebol]

Hello everyone,

First of all, I'm just starting to learn programming, and I'm a complete
newbie in Plan9, so please, be patient...  I was sad with the English
only spell checker, so I compiled international ispell with ape:

-Installed pdcurses.
-Patched term.c for termios.h, I used a linux patch (I don't have idea)
-Modified correct.c:827 (I had type conflicts).
		(void) fputs ((const char*)tok, logfile); 
-Edited local.h.generic for local.h:

	#define MINIMENU	/* Display a mini-menu at the bottom of the screen */
	#define USG		/* Define on System V or if term.c won't compile */
	#undef NO_FCNTL_H	/* Define if you get compile errors on fcntl.h */
	#define NO_MKSTEMP	/* Define if you get compile or link errors */
	#define CFLAGS	"-O -D_POSIX_SOURCE -D_BSD_EXTENSION"
	#define TERMLIB	"-lcurses"
	#define REGLIB	""
	#undef NO8BIT
	#define WORDS	"/usr/trebol/local/share/dict/words"
	
	#define LANGUAGES "{american,MASTERDICTS=american.med,HASHFILES=americanmed.hash,EXTRADICT=} {español}"
	/*
	 * Important directory paths.  If you change MAN45DIR from man5 to
	 * something else, you probably also want to set MAN45SECT and
	 * MAN45EXT (but not if you keep the man pages in section 5 and just
	 * store them in a different place).
	 */
	#define BINDIR	"/usr/trebol/local/bin"
	#define LIBDIR	"/usr/trebol/local/lib"
	#define MAN1DIR	"/usr/trebol/local/man/man1"
	#define MAN45DIR "/usr/trebol/local/man/man5"

I used http://www.datsi.fi.upm.es/~coes/espa~nol-1.7.tar.gz for Spanish
spell checking, untar it in ispell-3.3.02/languages/español and made
some changes:

-Changed all files and directories names to utf8 (acme don't work fine with the ~)
-Added a utf8 formatter to the aff file:

	altstringtype "utf8" "tex" ".txt"
	
	altstringchar   á    \'a
	altstringchar   Á    \'A
	altstringchar   é    \'e
	altstringchar   É    \'E
	altstringchar   í    \'i
	altstringchar   Í    \'I
	altstringchar   ñ    \'n
	altstringchar   Ñ    \'N
	altstringchar   ó    \'o
	altstringchar   Ó    \'O
	altstringchar   ú    \'u
	altstringchar   Ú    \'U
	altstringchar   ü    \"u
	altstringchar   Ü    \"U

-Edited Makefile, changed LANGUAGE to español and corrected paths:
	
		...
	PATHADDER	=	../../
	BUILDHASH	=	../../buildhash
	UNSQ		=	../../unsq
	FIX8BIT		=	../../fix8bit
		...
	LANGUAGE	=	español
		...
	eñe:
		sh eñes
		...
	../../munchlist -v -l ...

'make' works fine, but the deformatters must be compiled, 'make install'
expects executables but 'make all' creates .o files.  After 'make install'
the Spanish .aff and .hash files must be moved to the correct directory.

Well, ispell's normal mode works, but the suggestions and the line of the
misspelled word aren't showed (curses's problems?). Interactive and -a,
-l modes works fine.

The next problem was acme.  I had to change spout.c:63 for no ASCII characters:
	if(isalpharune(c))

Then aspell.  I made a version for ispell...  well this was for me a
nightmare, but I've learned a lot about rc.

#!/bin/rc

args=()
spellflags=()
for(x){
	switch($x){
	case -d*
		spellflags=($spellflags $x)
	case -p*
		spellflags=($spellflags $x)
	case -T*
		spellflags=($spellflags $x)
	case *
		args = ($args $x)
	}
}

dir = /mnt/wsys
if(! test -f $dir/cons)
	dir = /mnt/term/$dir
id=`{cat $dir/new/ctl}
id=$id(1)

if(~ $#args 1 && ~ $args /*){
	adir = `{basename -d $args}
	args = `{basename $args}
	echo 'name '^$adir^/-spell > $dir/$id/ctl
	cd $adir
}
if not {
	echo 'name '^`{pwd}^/-spell > $dir/$id/ctl
}

{
	echo noscroll
	if(~ $#args 0)
		for(j in `{$home/local/bin/acme/spout | sort -t: -u +2 | sort -t: +1.1n}){if(test `{echo -n $j | $home/local/bin/ispell -l $spellflags}){echo -n $j; echo -n $j | $home/local/bin/ispell -a $spellflags | awk -F: '/^&/{ORS=""; print $2}'; echo}} > $dir/$id/body
	if not for(i in $args)
		cat $i | for(j in `{$home/local/bin/acme/spout | sort -t: -u +2 | sort -t: +1.1n}){if(test `{echo -n $j | $home/local/bin/ispell -l $spellflags}){echo -n $i; echo -n $j; echo -n $j | $home/local/bin/ispell -a $spellflags | awk -F: '/^&/{ORS=""; print $2}'; echo}} > $dir/$id/body
	echo clean
}> $dir/$id/ctl


This works, and the output is like this:

	test:#0,#7:centeya centena
	test:#8,#14:camiño camilo, camino, cariño
	test:#15,#21:camion camino, camión

So you can click to visit the file in the word misspelled, and also see the suggestions.

With -dlanguage, -ppersonaldictionary and -Tformatter you can use all of
the international ispell's dictionaries. I put functions in my profile
for a easy use:

fn aispelles {$home/local/bin/acme/aispell -p$home/lib/pdict -Tutf8 -despañol $*}
fn aispellen {$home/local/bin/acme/aispell -p$home/lib/pdict_en -damerican $*}

And you can have '>> /personal/dictionary/path' to acme's tag (or commands
file's window) for adding words to your personal dictionary quickly.

The problem is the 'for' statement.  When the file grows a little
the script works VERY slow. And some times I have 'test: unexpected
operator/operand:' with this test expression.

I'm sure there is a faster (and more proper) way to make this right,
so I'll appreciate any help.

Regards,
trebol.

Re: [9fans] International Ispell in Plan9

[Mark van Atten]

That's interesting work---thanks!

Mark.

Re: [9fans] International Ispell in Plan9

[Nemo]

you could put it in sources, if not yet there.

[9fans] International Ispell in Plan9

[trebol]

On Thu, Apr 11, 2013 at 07:57:13PM +0200, Nemo wrote:
> you could put it in sources, if not yet there.


I want to put order in this mess before put it in sources.

I change the for loop to work in the output of ispell instead, and now
ispell works only one time in terse mode. The script is now much faster
thanks to the good design of awk and grep.

#!/bin/rc

rm -f /tmp/$pid^'.'aispell*

args=()
spellflags=()
for(x){
	switch($x){
	case -d*
		spellflags=($spellflags $x)
	case -p*
		spellflags=($spellflags $x)
	case -T*
		spellflags=($spellflags $x)
	case *
		args = ($args $x)
	}
}

dir = /mnt/wsys
if(! test -f $dir/cons)
	dir = /mnt/term/$dir
id=`{cat $dir/new/ctl}
id=$id(1)

if(~ $#args 1 && ~ $args /*){
	adir = `{basename -d $args}
	args = `{basename $args}
	echo 'name '^$adir^/-spell > $dir/$id/ctl
	cd $adir
}
if not {
	echo 'name '^`{pwd}^/-spell > $dir/$id/ctl
}

{
	echo noscroll
	if(~ $#args 0){
		cat > /tmp/$pid^'.'aispell0; i = /tmp/$pid^'.'aispell0; winname = `{cat /mnt/acme/$winid/tag | awk '{print $1}'}; for(j in `{cat $i | $home/local/bin/ispell -a $spellflags | awk '/^[&#]/{gsub(/ /,"_"); print}'}){$home/local/bin/acme/spout $i | grep `{echo $j | awk -F_ '{print $2}'} | awk -F: '{OFS=":";$1 = "'$winname'"; print}' >> /tmp/$pid^'.'aispell } ; sort -u /tmp/$pid^'.'aispell > $dir/$id/body; rm -f /tmp/$pid^'.'aispell*
	}
	if not for(i in $args){
		for(j in `{cat $i | $home/local/bin/ispell -a $spellflags | awk '/^[&#]/{gsub(/ /,"_"); print}'}){$home/local/bin/acme/spout $i | grep `{echo $j | awk -F_ '{print $2}'} >> /tmp/$pid^'.'aispell } ; sort -u /tmp/$pid^'.'aispell > $dir/$id/body; rm -f /tmp/$pid^'.'aispell
	}
	echo clean
}> $dir/$id/ctl


Now you can use it in the tag line to spell check the dot, and the
output begins with the name of the window, so if you select all the
window's body, you can spell check it without save it with the same
commodity. Of course the addresses of the misspelled words don't works
with a common selection.

To make this work, I need to know how to get the dot address within
the script.  Also the functions don't work in acme, but a similar script
works. Why?

fn aispellen {$home/local/bin/acme/aispell -p$home/lib/pdict_en -damerican $*}

Any help?

Re: [9fans] International Ispell in Plan9

[Francisco J Ballesteros]

look under /acme for examples. 

On Apr 12, 2013, at 2:21 PM, trebol <trebol55555@yahoo.es> wrote:

> On Thu, Apr 11, 2013 at 07:57:13PM +0200, Nemo wrote:
>> you could put it in sources, if not yet there.
> 
> 
> I want to put order in this mess before put it in sources.
> 
> I change the for loop to work in the output of ispell instead, and now
> ispell works only one time in terse mode. The script is now much faster
> thanks to the good design of awk and grep.
> 
> #!/bin/rc
> 
> rm -f /tmp/$pid^'.'aispell*
> 
> args=()
> spellflags=()
> for(x){
>    switch($x){
>    case -d*
>        spellflags=($spellflags $x)
>    case -p*
>        spellflags=($spellflags $x)
>    case -T*
>        spellflags=($spellflags $x)
>    case *
>        args = ($args $x)
>    }
> }
> 
> dir = /mnt/wsys
> if(! test -f $dir/cons)
>    dir = /mnt/term/$dir
> id=`{cat $dir/new/ctl}
> id=$id(1)
> 
> if(~ $#args 1 && ~ $args /*){
>    adir = `{basename -d $args}
>    args = `{basename $args}
>    echo 'name '^$adir^/-spell > $dir/$id/ctl
>    cd $adir
> }
> if not {
>    echo 'name '^`{pwd}^/-spell > $dir/$id/ctl
> }
> 
> {
>    echo noscroll
>    if(~ $#args 0){
>        cat > /tmp/$pid^'.'aispell0; i = /tmp/$pid^'.'aispell0; winname = `{cat /mnt/acme/$winid/tag | awk '{print $1}'}; for(j in `{cat $i | $home/local/bin/ispell -a $spellflags | awk '/^[&#]/{gsub(/ /,"_"); print}'}){$home/local/bin/acme/spout $i | grep `{echo $j | awk -F_ '{print $2}'} | awk -F: '{OFS=":";$1 = "'$winname'"; print}' >> /tmp/$pid^'.'aispell } ; sort -u /tmp/$pid^'.'aispell > $dir/$id/body; rm -f /tmp/$pid^'.'aispell*
>    }
>    if not for(i in $args){
>        for(j in `{cat $i | $home/local/bin/ispell -a $spellflags | awk '/^[&#]/{gsub(/ /,"_"); print}'}){$home/local/bin/acme/spout $i | grep `{echo $j | awk -F_ '{print $2}'} >> /tmp/$pid^'.'aispell } ; sort -u /tmp/$pid^'.'aispell > $dir/$id/body; rm -f /tmp/$pid^'.'aispell
>    }
>    echo clean
> }> $dir/$id/ctl
> 
> 
> Now you can use it in the tag line to spell check the dot, and the
> output begins with the name of the window, so if you select all the
> window's body, you can spell check it without save it with the same
> commodity. Of course the addresses of the misspelled words don't works
> with a common selection.
> 
> To make this work, I need to know how to get the dot address within
> the script.  Also the functions don't work in acme, but a similar script
> works. Why?
> 
> fn aispellen {$home/local/bin/acme/aispell -p$home/lib/pdict_en -damerican $*}
> 
> Any help?

Re: [9fans] International Ispell in Plan9

[erik quanstrom]

> {
> 	echo noscroll
> 	if(~ $#args 0){
> 		cat > /tmp/$pid^'.'aispell0; i = /tmp/$pid^'.'aispell0; winname = `{cat /mnt/acme/$winid/tag | awk '{print $1}'}; for(j in `{cat $i | $home/local/bin/ispell -a $spellflags | awk '/^[&#]/{gsub(/ /,"_"); print}'}){$home/local/bin/acme/spout $i | grep `{echo $j | awk -F_ '{print $2}'} | awk -F: '{OFS=":";$1 = "'$winname'"; print}' >> /tmp/$pid^'.'aispell } ; sort -u /tmp/$pid^'.'aispell > $dir/$id/body; rm -f /tmp/$pid^'.'aispell*
> 	}
> 	if not for(i in $args){
> 		for(j in `{cat $i | $home/local/bin/ispell -a $spellflags | awk '/^[&#]/{gsub(/ /,"_"); print}'}){$home/local/bin/acme/spout $i | grep `{echo $j | awk -F_ '{print $2}'} >> /tmp/$pid^'.'aispell } ; sort -u /tmp/$pid^'.'aispell > $dir/$id/body; rm -f /tmp/$pid^'.'aispell
> 	}
> 	echo clean
> }> $dir/$id/ctl

the rest of the script is nicely formatted.  but it looks
like this bit could use some formatting.  remember, you
get a free newline after { and |.

there is some opportunity to simplify, too.  for example
	cat /mnt/acme/$winid/tag | awk '{print $1}'
is more simply
	sed 's/ .*//g' < /mnt/acme/$winid/tag
i see several places one could replace awk with sed and cat with
redirection.

and i think further simplification is possible by using
{} instead of the temporary file dance.  and i'd imagine
that that's where the bug is.

- erik

[9fans] International Ispell in Plan9

[trebol]

Thanks for the help erik!

This is the best I made for now...

/////////////////////////////////////
/////////////////////////////////////

#!/bin/rc

rm -f /tmp/$pid^'.'aispell*

args=()
spellflags=()
for(x){
	switch($x){
	case -d*
		spellflags=($spellflags $x)
	case -p*
		spellflags=($spellflags $x)
	case -T*
		spellflags=($spellflags $x)
	case *
		args = ($args $x)
	}
}

dir = /mnt/wsys
if(! test -f $dir/cons)
	dir = /mnt/term/$dir
id=`{cat $dir/new/ctl}
id=$id(1)

if(~ $#args 1 && ~ $args /*){
	adir = `{basename -d $args}
	args = `{basename $args}
	echo 'name '^$adir^/-spell > $dir/$id/ctl
	cd $adir
}
if not {
	echo 'name '^`{pwd}^/-spell > $dir/$id/ctl
}

{
	echo noscroll
	if(~ $#args 0){
		cat > /tmp/$pid^'.'aispell0
		i = /tmp/$pid^'.'aispell0
		winname = `{sed 's/ .*//g' < /mnt/acme/$winid/tag}
		if(~ $winname '') winname = nonamedwindow
		for(j in `{$home/local/bin/ispell -a $spellflags < $i | grep '^[&#]' | sed 's/ /_/g'}){
			{cat $i; echo} | $home/local/bin/acme/spout | # spout needs \n
			grep `{echo $j |
			awk -F_ '{print $2}'} |
			awk -F: '{OFS=":";$1 = "'$i'"; print}' >> /tmp/$pid^'.'aispell 
		} 
		sort -u /tmp/$pid^'.'aispell > $dir/$id/body
		rm -f /tmp/$pid^'.'aispell*
	}
	if not for(i in $args){
		for(j in `{ $home/local/bin/ispell -a $spellflags < $i | grep '^[&#]' | sed 's/ /_/g'}){
		{cat $i; echo} | $home/local/bin/acme/spout |
		grep `{echo $j | awk -F_ '{print $2}'} |
		awk -F: '{OFS=":";$1 = "'$i'"; print}'>> /tmp/$pid^'.'aispell
		}
		sort -u /tmp/$pid^'.'aispell > $dir/$id/body
		rm -f /tmp/$pid^'.'aispell
	}
	echo clean
}> $dir/$id/ctl


/////////////////////////////////////
/////////////////////////////////////

Is 
	grep '^[&#]' | sed 's/ /_/g' 

better than

	awk '/^[&#]/{gsub(/ /,"_"); print}'
?

I can use sed instead of awk in 
	awk -F: '{OFS=":";$1 = "'$i'"; print}'

but if the tag name contains the character used in sed as '/' it will
cause errors.  I discovered working in this script that acme has problems
with file names containing whitespaces (I only can open it at startup
and if you click in the tag strange things happens...) so a solution would
be:

sed 's ^[^:]*: '$winname': g'

> and i think further simplification is possible by using
> {} instead of the temporary file dance.

I don't have the knowledge to make a better redirection work with the
standard input sent to the script.  Specially with the output of the
statements inside the for loop.  The file funky dance was the only way I
found to make the work. 

> and i'd imagine that that's where the bug is.

The script works fine, 'aispell /some/file', 'aispell < /some/file' and
executing  '> aispell' in a acme's window with some text selected works as
expected.  But executing '> aispelles' in a acme's window, where aispelles
is a function:

	'fn aispelles {/aispell/path -despañol -Tutf8 -p/personal/dictionary/path $*}' 

doesn't works.  But this function in win or outside acme works fine
('aispelles < /some/file' and 'aispelles /some/file'), so I
don't know what is the problem.

Also, I corrected a bug related with spout, it needs \n previous to EOF, so
the last line in a input without \n now doesn't stay out of the check.

I have looked in the acme scripts a lot and I can't find any example of
getting the dot address, only in c code.  If anyone knows how to make it
in rc, I will be very grateful.  I think with the feature of selecting a address
in the output of the script executed to any selected text in a window,
and pointing the cursor in the right position, the work will
be finish.

I'm learning a lot about Plan9, acme, rc, awk, sed ... perhaps I should have to wait
until I have the skills and knowledge about programming and Plan9 before
posting anything like this in the list, but I thought this could be useful
for someone.

Regards,
trebol.

[9fans] International Ispell in Plan9

[trebol]

Hello everyone.

This script works very fine.  I will take a look at sources, and learn
the proper way to share this.

There is a lot of dictionaries you can use with ispell:

http://fmg-www.cs.ucla.edu/geoff/ispell-dictionaries.html


I will make a package with ispell and the American and British
dictionaries, and a separate package with the Spanish dictionary.
I'm starting to learn Portuguese, so I will try to make a package for
this language too, but I think it would be great if a native speaker
of each language makes and tests the dictionaries.  The work is easy,
just make a utf8 formatter like that I showed you in the first mail I
send with this subject.

With the dot's address I can fix the output to make it work with
arbitrarily selections.  I'm still looking for some way to get the
dot's address of a window within a rc script, but I think that the
proper way it's going to be keep learning C (just 3th chapter of K&R
for now...) and make a simple C program to make the work and put it in
/acme/bin for future use in scripts, and maybe substitute the script
with a faster program.

Regards,
trebol.

//////////////////////////
//////////////////////////

#!/bin/rc

rm -f /tmp/$pid^'.'aispell

args=()
spellflags=()
for(x){
	switch($x){
	case -d*
		spellflags=($spellflags $x)
	case -p*
		spellflags=($spellflags $x)
	case -T*
		spellflags=($spellflags $x)
	case *
		args = ($args $x)
	}
}

dir = /mnt/wsys
if(! test -f $dir/cons)
	dir = /mnt/term/$dir
id=`{cat $dir/new/ctl}
id=$id(1)

#if(~ $#args 1 && ~ $args /*){
#	adir = `{basename -d $args}
#	args = `{basename $args}
#	echo 'name '^$adir^/-spell > $dir/$id/ctl
#	cd $adir
#}
#if not {
	echo 'name '^`{pwd}^/-spell > $dir/$id/ctl
#}

{
	echo noscroll
	if(~ $#args 0){
		cat > /tmp/$pid^'.'aispell
		args = /tmp/$pid^'.'aispell
		pipe = 1
	}
	for(i in $args){
			name = $i
			if(~ $pipe 1){
				name = `{sed 's/ .*//g' < /mnt/acme/$winid/tag}
				if(~ name '') name = nonamedwindow
			}
		for(j in `{{cat $i; echo} | $home/local/bin/acme/spout | sort -t: -u +2 | sed 's/$/\!/g' | $home/local/bin/ispell -a $spellflags | grep '^[&#]' | sed 's/ /_/g'}){
		# {cat $i; echo} is for spout, needs \n. I want make a list of lines, so j can't have spaces
			miss = `{ echo $j | awk -F_ '{print $2}'}
			sugg = `{ echo $j | sed 's/^.*://g'} # I can't put grep -v '^#' here...
			{cat $i; echo} |
			$home/local/bin/acme/spout |
			grep '.*:'$miss'$' |
			sed 's/$/ '$sugg'/g' | # If I put grep -v '^#' above, this sed cuts output, I don't know why ...
			sed 's/#_.*$//g' |
			sed 's/_/ /g' |  # If I put sed 's/_/ /g' above, variables don't work in sed.  Again, I don't know why...
			sed s',^,'$name',g' > $dir/$id/body
		}

	}
	rm -f /tmp/$pid^'.'aispell
	echo clean
}> $dir/$id/ctl

[9fans] International Ispell in Plan9

[trebol]

The script don't work, and has serious mistakes in its approach.  I will
fix it soon.

Regards,
trebol.

[9fans] International Ispell in Plan9

[trebol]

> And you can have '>> /personal/dictionary/path' [...]

Sorry, this must be 'echo >> /personal/dictionary/path' and make a 2-1 mouse
chord.

Regards,
trebol.