* any partial or rudamentary documentation of new nnselect work somewhere? @ 2017-05-06 22:39 Harry Putnam 2017-05-07 0:10 ` Andrew Cohen 0 siblings, 1 reply; 11+ messages in thread From: Harry Putnam @ 2017-05-06 22:39 UTC (permalink / raw) To: ding I'd like to get my hands on some bit of docu for the new nnselect work that is being done... I realize it is not done and won't be for some time. But I need a start of some kind. Such things as the search syntax and something explaining the basics of how the search terms need to work with many backends. That is, some big picture explanations and possibly as many actual examples as anyone has time to write. Something as rough as some of the experts working on nnselect might save there test command lines off to a file ... even with no explanatory remarks and make that available somewhere. I've followed the discussion to the best of my ability... but mostly its over my head. I would still like to do some testing and I suspect the work needs to have a few non-expert testers at some point to make sure the new work is usable to some of us dim wits too. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: any partial or rudamentary documentation of new nnselect work somewhere? 2017-05-06 22:39 any partial or rudamentary documentation of new nnselect work somewhere? Harry Putnam @ 2017-05-07 0:10 ` Andrew Cohen 2017-05-10 20:49 ` Harry Putnam 0 siblings, 1 reply; 11+ messages in thread From: Andrew Cohen @ 2017-05-07 0:10 UTC (permalink / raw) To: ding Dear Harry (and others) >>>>> "Harry" == Harry Putnam <reader@newsguy.com> writes: Harry> I'd like to get my hands on some bit of docu for the new Harry> nnselect work that is being done... I realize it is not done Harry> and won't be for some time. Actually, almost done :) Harry> But I need a start of some kind. Such things as the search Harry> syntax and something explaining the basics of how the search Harry> terms need to work with many backends. That is, some big Harry> picture explanations and possibly as many actual examples as Harry> anyone has time to write. First, nnselect and the new universal search syntax are two different things. The feature branch in git feature/gnus-select contains the nnselect work but not the new search language. So if you check out this branch you should be able to continue working with essentially no change in anything: if all you are doing is searching that will work the same way as before (it will use the nnselect backend behind the scenes but shouldn't behave markedly differently from the way things behaved earlier). It would be great if you could try this out and report any bugs. MAKE A BACKUP OF .newsrc.eld JUST IN CASE! nnselect allows doing lots of other things as well, but you can worry about those later once there is some documentation in place. Or you can ask me and I can send you some recipes. Harry> Something as rough as some of the experts working on nnselect Harry> might save there test command lines off to a file ... even Harry> with no explanatory remarks and make that available Harry> somewhere. That's easy---there are no special commands :) For basic search just "G G" as usual. Or if you want to create a permanent search group (that is, one that behaves like any other group but the selected articles are the result of a search query) use "G g". Harry> I've followed the discussion to the best of my ability... but Harry> mostly its over my head. Harry> I would still like to do some testing and I suspect the work Harry> needs to have a few non-expert testers at some point to make Harry> sure the new work is usable to some of us dim wits too. Definitely! First I would like to make sure nnselect isn't going to do anything bad (not likely, but needs testing). This should be easy since users don't need to learn anything new to test the basic structure. Once that is working correctly we will merge in the universal search language that Eric has developed. This will require users to change the syntax of their searches and might have a bit of a learning curve (but only a bit---the language is clear and simple so it shouldn't pose any difficulties). Best, Andy ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: any partial or rudamentary documentation of new nnselect work somewhere? 2017-05-07 0:10 ` Andrew Cohen @ 2017-05-10 20:49 ` Harry Putnam 2017-05-11 0:01 ` Andrew Cohen 0 siblings, 1 reply; 11+ messages in thread From: Harry Putnam @ 2017-05-10 20:49 UTC (permalink / raw) To: ding Andrew Cohen <cohen@bu.edu> writes: > Harry> Something as rough as some of the experts working on nnselect > Harry> might save there test command lines off to a file ... even > Harry> with no explanatory remarks and make that available > Harry> somewhere. > That's easy---there are no special commands :) For basic search just "G > G" as usual. Or if you want to create a permanent search group (that is, > one that behaves like any other group but the selected articles are the > result of a search query) use "G g". [I really don't want to bog down or belabor something when such a vigorous and productive spate of coding is underway, so maybe you can just post urls or direct me to the proper place to get started] In emacs from source of a few days ago (and with feature/gnus-select pulled in too): OK, I guess I'm thoroughly lost here... Your comments seem to indicate there is existing nnselect doucumentation some where. Or at least that the functionality of existing code is known to gnus. At the top of gnus manual... (The one that comes with emacs sources when one sets up an emacs branch from git repo) That manual says it belongs to version 5.13 ... but isn't that quite old? Anyway: Using regex search `s' and entering `nnselect', there are no hits at all. In gnus... Entering a group and pressing `G G'... I am prompted for an article number. Surely that is not where one enters search terms, since it is asking for an article number. But even if I were to find something explaining how to set up and use nnselect... I'd still need to know how to enter a search. That is, the syntax. I'm guessing one does not use a regex like in `M-s' in group buffer, type searches. If this information is readily available please just respond with RTFM or the like depending on where it is. I don't want to waste your time. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: any partial or rudamentary documentation of new nnselect work somewhere? 2017-05-10 20:49 ` Harry Putnam @ 2017-05-11 0:01 ` Andrew Cohen 2017-05-11 13:07 ` Harry Putnam 0 siblings, 1 reply; 11+ messages in thread From: Andrew Cohen @ 2017-05-11 0:01 UTC (permalink / raw) To: ding Dear Harry: >>>>> "Harry" == Harry Putnam <reader@newsguy.com> writes: Harry> Andrew Cohen <cohen@bu.edu> writes: Something as rough as Harry> some of the experts working on nnselect might save there test Harry> command lines off to a file ... even with no explanatory Harry> remarks and make that available somewhere. >> That's easy---there are no special commands :) For basic search >> just "G G" as usual. Or if you want to create a permanent search >> group (that is, one that behaves like any other group but the >> selected articles are the result of a search query) use "G g". [...] Harry> In emacs from source of a few days ago (and with Harry> feature/gnus-select pulled in too): Harry> OK, I guess I'm thoroughly lost here... Your comments seem to Harry> indicate there is existing nnselect doucumentation some Harry> where. Or at least that the functionality of existing code is Harry> known to gnus. [...] Sorry, I hadn't realized that you weren't using search previously. First there are two "kinds" of searching---in a summary buffer you can search (in a variety of ways, including the regex search you were trying); or you can query servers to find articles matching certain search terms. In this thread we are only talking about the latter search facility. It is usually referred to as "nnir" searching. It is documented in the gnus manual under "Searching" (go to info, select gnus, and select the searching node). This should give you the information you need to start searching (you might have to do some configuration depending on what backends you are using). In the various threads going on currently on this mailing list we are talking about changes and extensions to this facility. There are two independent things: 1. Introduce a new backend, nnselect, that among other uses will replace the part of nnir that manages the "gnus" related handling of articles found as part of a search. 2. Introduce a new, universal, query language for searching for articles. Item 1. can be tested without modifying anything (that is, if you have nnir searching set up as in the manual then using the new code from feature/gnus-select should mostly just work without any new configuration). Item 2. is in a separate feature branch---pulling in feature/gnus-select will NOT currently alter the way searches are done. So the first step in any testing is to make sure that nnir-based searching is working for you PRIOR to testing the new stuff. I would suggest reading the manual to get started. Best, Andy ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: any partial or rudamentary documentation of new nnselect work somewhere? 2017-05-11 0:01 ` Andrew Cohen @ 2017-05-11 13:07 ` Harry Putnam 2017-05-12 1:18 ` Eric Abrahamsen 0 siblings, 1 reply; 11+ messages in thread From: Harry Putnam @ 2017-05-11 13:07 UTC (permalink / raw) To: ding Andrew Cohen <cohen@bu.edu> writes: [...] > Sorry, I hadn't realized that you weren't using search previously. First > there are two "kinds" of searching---in a summary buffer you can search > (in a variety of ways, including the regex search you were trying); or > you can query servers to find articles matching certain search terms. In > this thread we are only talking about the latter search facility. It is > usually referred to as "nnir" searching. It is documented in the gnus > manual under "Searching" (go to info, select gnus, and select the > searching node). This should give you the information you need to start > searching (you might have to do some configuration depending on what > backends you are using). [...] Sorry, I should have explained the massive holes in my knowledge of gnus. Some yrs ago now, I wrote my own search engine in perl... I mean just a perl script that searched for regex thru thousands of gmane messages that I had downloaded with the agent. So much downloading that I was kicked off gmane a few times... probably looked like something sinister. I could search on exact code in perl groups for example ... and return exact bits of code with careful use of perl regex. Later on I added more (perl) code that created a directory and filled it with symlinks to the actual messages where my regex found hits. I could then pull that directory into gnus as an ephemeral group or nndoc inside gnus. All very clunky and probably duplicating functionality already in gnus but in a primitive and homeboy way. Also very slow. Hardcore regex searching of that many messages is rather slow. And I'm not a good enough perl coder to know how to speed it up. The only upside is that it is very exact. At the high point I had used the agent to collect well over 200,000 messages ... no longer remember how many, but the slowness of searches eventually led me to fade out on all that search activity. Not to mention that I quit getting barred on gmane too. So haven't really worked at it for several years. And never really learned the gnus capabilities in nnir, namzu, swish, and etc. All the new efforts are pretty exciting and has gotten me interested once again. Many thanks for the nice summaries and big picture guidance. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: any partial or rudamentary documentation of new nnselect work somewhere? 2017-05-11 13:07 ` Harry Putnam @ 2017-05-12 1:18 ` Eric Abrahamsen 2017-05-13 13:44 ` Harry Putnam 0 siblings, 1 reply; 11+ messages in thread From: Eric Abrahamsen @ 2017-05-12 1:18 UTC (permalink / raw) To: ding Harry Putnam <reader@newsguy.com> writes: > Andrew Cohen <cohen@bu.edu> writes: > > [...] > >> Sorry, I hadn't realized that you weren't using search previously. First >> there are two "kinds" of searching---in a summary buffer you can search >> (in a variety of ways, including the regex search you were trying); or >> you can query servers to find articles matching certain search terms. In >> this thread we are only talking about the latter search facility. It is >> usually referred to as "nnir" searching. It is documented in the gnus >> manual under "Searching" (go to info, select gnus, and select the >> searching node). This should give you the information you need to start >> searching (you might have to do some configuration depending on what >> backends you are using). > > [...] > > Sorry, I should have explained the massive holes in my knowledge of gnus. > > Some yrs ago now, I wrote my own search engine in perl... I mean just > a perl script that searched for regex thru thousands of gmane messages > that I had downloaded with the agent. So much downloading that I was > kicked off gmane a few times... probably looked like something > sinister. > > I could search on exact code in perl groups for example ... and return > exact bits of code with careful use of perl regex. > > Later on I added more (perl) code that created a directory and filled > it with symlinks to the actual messages where my regex found hits. I > could then pull that directory into gnus as an ephemeral group or > nndoc inside gnus. > > All very clunky and probably duplicating functionality already in gnus > but in a primitive and homeboy way. Also very slow. It does duplicate what's possible in Gnus (this could be done even with Gnus as it stands now), using an external indexing program like namazu or notmuch. The one big caveat would be that these programs almost certainly wouldn't index perl code correctly -- I haven't tried it, but I'd be very surprised if they did it right. The generalized search functionality that I'm working on (built on Andy's nnselect branch) would let you create all kinds of weird search engine behavior. In theory it wouldn't be hard at all to create a "grep" mixin class for the indexed search engines, that accepted an additional "grep:" key, and ran the grep over the results of the initial search. Ie, you could enter the search string: subject:"having a problem" grep:"my ($smith, $jones) = @a;" since:3m The subject: and since: keys would be passed to the underlying search engine (notmuch or mairix, etc), which would return a first round of results very quickly. The results are in the form of filenames, so the grep: key could then be used to do a second pass over those files. A best-of-both-worlds situation. That sounds pretty useful. Maybe there's no need for a mixin class at all; this could be a standard feature of the indexed search class. If grep wasn't available on the system, you'd just get a polite note to that effect. Actually, there's already a regexp syntax: surrounding text with forward slashes. Search engines that don't handle regexps could simply transform "body:/(.*)/" into "grep:(.*)"... Hmmm... ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: any partial or rudamentary documentation of new nnselect work somewhere? 2017-05-12 1:18 ` Eric Abrahamsen @ 2017-05-13 13:44 ` Harry Putnam 2017-05-14 2:48 ` Eric Abrahamsen 0 siblings, 1 reply; 11+ messages in thread From: Harry Putnam @ 2017-05-13 13:44 UTC (permalink / raw) To: ding Eric Abrahamsen <eric@ericabrahamsen.net> writes: > Harry Putnam <reader@newsguy.com> writes: [...] >> I could search on exact code in perl groups for example ... and return >> exact bits of code with careful use of perl regex. >> >> Later on I added more (perl) code that created a directory and filled >> it with symlinks to the actual messages where my regex found hits. I >> could then pull that directory into gnus as an ephemeral group or >> nndoc inside gnus. >> >> All very clunky and probably duplicating functionality already in gnus >> but in a primitive and homeboy way. Also very slow. > > It does duplicate what's possible in Gnus (this could be done even with > Gnus as it stands now), using an external indexing program like namazu > or notmuch. The one big caveat would be that these programs almost > certainly wouldn't index perl code correctly -- I haven't tried it, but > I'd be very surprised if they did it right. That was roughly my thinking at the time...(about indexing) I considered trying to incorporate one or another of the indexing search tools available. But, first off, I didn't know how to do that... but might have fumbled my way thru. But, second and more important... indexing perl or other kinds of code; I new would be a loser. And that was my primary objective at the time. Back then there was a tool called glimpse that was still under development (although, barely) that does have a near regex capability and is also an indexing tool... but once again the indexing doesn't do well with code. So, decided to put up with slowness of full-fledged perl regex searching. Decided indexing just didn't go with the kind of searchs I was interested in. You would know vastly more than I about the capabilities of any of the search tools available now. But I believe you are right in thinking they would not index code very well at all. > The generalized search functionality that I'm working on (built on > Andy's nnselect branch) would let you create all kinds of weird search > engine behavior. In theory it wouldn't be hard at all to create a "grep" > mixin class for the indexed search engines, that accepted an additional > "grep:" key, and ran the grep over the results of the initial search. > Ie, you could enter the search string: > > subject:"having a problem" grep:"my ($smith, $jones) = @a;" since:3m > > The subject: and since: keys would be passed to the underlying search > engine (notmuch or mairix, etc), which would return a first round of > results very quickly. The results are in the form of filenames, so the > grep: key could then be used to do a second pass over those files. A > best-of-both-worlds situation. > > That sounds pretty useful. Maybe there's no need for a mixin class at > all; this could be a standard feature of the indexed search class. If > grep wasn't available on the system, you'd just get a polite note to > that effect. > Actually, there's already a regexp syntax: surrounding text with forward > slashes. Search engines that don't handle regexps could simply transform > "body:/(.*)/" into "grep:(.*)"... > > Hmmm... Yes, yes, and away you go... That two phase bit sounds really useful. I am really sorry to be of no help at all. If you fellows have any low level scut work that needs doing, and I do mean `low level'... that is, no coding required. I realize that is a piss poor offer, and there probably is nothing like that involved... but still ... just in case .. the offer stands. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: any partial or rudamentary documentation of new nnselect work somewhere? 2017-05-13 13:44 ` Harry Putnam @ 2017-05-14 2:48 ` Eric Abrahamsen 2017-05-14 13:21 ` Harry Putnam 2017-05-14 13:24 ` Harry Putnam 0 siblings, 2 replies; 11+ messages in thread From: Eric Abrahamsen @ 2017-05-14 2:48 UTC (permalink / raw) To: ding Harry Putnam <reader@newsguy.com> writes: > You would know vastly more than I about the capabilities of any of the > search tools available now. But I believe you are right in thinking > they would not index code very well at all. "Vastly" is an overstatement :) But I think we can assume it's not going to work. [...] >> That sounds pretty useful. Maybe there's no need for a mixin class at >> all; this could be a standard feature of the indexed search class. If >> grep wasn't available on the system, you'd just get a polite note to >> that effect. > >> Actually, there's already a regexp syntax: surrounding text with forward >> slashes. Search engines that don't handle regexps could simply transform >> "body:/(.*)/" into "grep:(.*)"... >> >> Hmmm... > > Yes, yes, and away you go... That two phase bit sounds really > useful. I am really sorry to be of no help at all. > > If you fellows have any low level scut work that needs doing, and I do > mean `low level'... that is, no coding required. > > I realize that is a piss poor offer, and there probably is nothing > like that involved... but still ... just in case .. the offer stands. Thinking about it further, I think automatically translating regexps into "grep:" might be going overboard, and would introduce unnecessary complications. But providing the grep functionality itself still sounds like a good idea. And thanks for the offer of help! It's true there's not a whole lot to be done at this point, just a bit of leftover functionality, then tests and docs. What would be most useful is if someone could explain git better to me so that the rebase process was less bewildering :( ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: any partial or rudamentary documentation of new nnselect work somewhere? 2017-05-14 2:48 ` Eric Abrahamsen @ 2017-05-14 13:21 ` Harry Putnam 2017-05-14 13:24 ` Harry Putnam 1 sibling, 0 replies; 11+ messages in thread From: Harry Putnam @ 2017-05-14 13:21 UTC (permalink / raw) To: ding Eric Abrahamsen <eric@ericabrahamsen.net> writes: > What would be most useful is if someone could explain git > better to me so that the rebase process was less bewildering :( If no one responds to that... (and you've no doubt considered this) But, you could take the query to gmane.comp.version-control.git There are some heavy hitters there that can probably explain things thouroughly. The name of the group does not suggest it is a devel group but that does appear to be mostly what is going on there. None the less, I've seen many detailed questions answered there. Well out of my league. But there are many true experts there that appear willing to answer complicated question. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: any partial or rudamentary documentation of new nnselect work somewhere? 2017-05-14 2:48 ` Eric Abrahamsen 2017-05-14 13:21 ` Harry Putnam @ 2017-05-14 13:24 ` Harry Putnam 2017-05-15 3:30 ` Eric Abrahamsen 1 sibling, 1 reply; 11+ messages in thread From: Harry Putnam @ 2017-05-14 13:24 UTC (permalink / raw) To: ding Eric Abrahamsen <eric@ericabrahamsen.net> writes: > Thinking about it further, I think automatically translating regexps > into "grep:" might be going overboard, and would introduce unnecessary > complications. But providing the grep functionality itself still sounds > like a good idea. Anything that functions grep like for that second pass would really be great. I have no idea what coding that would look like, but hope it is not too daunting or something that will bog things down. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: any partial or rudamentary documentation of new nnselect work somewhere? 2017-05-14 13:24 ` Harry Putnam @ 2017-05-15 3:30 ` Eric Abrahamsen 0 siblings, 0 replies; 11+ messages in thread From: Eric Abrahamsen @ 2017-05-15 3:30 UTC (permalink / raw) To: ding Harry Putnam <reader@newsguy.com> writes: > Eric Abrahamsen <eric@ericabrahamsen.net> writes: > >> Thinking about it further, I think automatically translating regexps >> into "grep:" might be going overboard, and would introduce unnecessary >> complications. But providing the grep functionality itself still sounds >> like a good idea. > > Anything that functions grep like for that second pass would really be > great. > > I have no idea what coding that would look like, but hope it is not > too daunting or something that will bog things down. No, it shouldn't be too difficult. The guts of the code already exist in the find-grep engine, which I kind of doubt anyone's using, but which can be refactored to use this same mechanism. ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2017-05-15 3:30 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-05-06 22:39 any partial or rudamentary documentation of new nnselect work somewhere? Harry Putnam 2017-05-07 0:10 ` Andrew Cohen 2017-05-10 20:49 ` Harry Putnam 2017-05-11 0:01 ` Andrew Cohen 2017-05-11 13:07 ` Harry Putnam 2017-05-12 1:18 ` Eric Abrahamsen 2017-05-13 13:44 ` Harry Putnam 2017-05-14 2:48 ` Eric Abrahamsen 2017-05-14 13:21 ` Harry Putnam 2017-05-14 13:24 ` Harry Putnam 2017-05-15 3:30 ` Eric Abrahamsen
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).