* Re: Gdbm module [not found] <CAKc7PVAz=zfrFapifcJudnMqgscNudC5S4S6ym7AWfFFORoROQ@mail.gmail.com> @ 2017-02-21 17:25 ` Bart Schaefer 2017-02-21 19:52 ` Sebastian Gniazdowski 0 siblings, 1 reply; 6+ messages in thread From: Bart Schaefer @ 2017-02-21 17:25 UTC (permalink / raw) To: Sebastian Gniazdowski; +Cc: zsh-workers On Feb 21, 12:21pm, Sebastian Gniazdowski wrote: } } Do you think commiting "zgdbmclear" I've sent doesn't have sense? I would prefer a bit more thought be put into the design. There's no particular problem with the implementation underneath, but: The whole point of "ztie -d ..." and having the modules named with the /db/ piece in the middle is so that a generic interface to multiple database backends can be implemented. "ztie" and "zuntie" happen to be in db_gdbm.c for now because there is no other such module, but the intention was that there would be a base "zsh/db" module of which zsh/db/gdbm was one implementation. In keeping with that, there shouldn't be commands named "zgdbm..." and there should be no "zgdbm_tied" parameter -- there should instead be a single array or hash in the (so far nonexistent) base module to return all the ztie'd names, and if there are other ops needed than z(un)tie then something like "zdb path ...", "zdb clear ...", etc. to give a consistent front to whatever backend has been passed to "ztie -d". (Or option names/letters instead of subcommands, to be more consistent with most builtins, but you get the idea.) This may mean that there are some ops that either we don't provide for any backend because they're too specific to one, or that fail for some backends. E.g. "zdb path" doesn't make sense for a network-connected database; maybe it returns a URL in that case, or we should consider that the -f FILENAME argument of ztie is not sufficiently general, and "zdb path" would just return back that argument of ztie. None of this means we need to rush off to implement zsh/db/sqlite or something, but we should be thinking about it when extending db/gdbm. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Gdbm module 2017-02-21 17:25 ` Gdbm module Bart Schaefer @ 2017-02-21 19:52 ` Sebastian Gniazdowski 2017-02-22 4:41 ` Bart Schaefer 0 siblings, 1 reply; 6+ messages in thread From: Sebastian Gniazdowski @ 2017-02-21 19:52 UTC (permalink / raw) To: zsh-workers On Tue, Feb 21, 2017, at 09:25 AM, Bart Schaefer wrote: > On Feb 21, 12:21pm, Sebastian Gniazdowski wrote: > } > } Do you think commiting "zgdbmclear" I've sent doesn't have sense? > > I would prefer a bit more thought be put into the design. There's no > particular problem with the implementation underneath, but: > > The whole point of "ztie -d ..." and having the modules named with the > /db/ piece in the middle is so that a generic interface to multiple > database backends can be implemented. "ztie" and "zuntie" happen to > be in db_gdbm.c for now because there is no other such module, but > the intention was that there would be a base "zsh/db" module of which > zsh/db/gdbm was one implementation. The dream of general ztie.. I saw how difficult it is when writing. Now it seems totally possible, but when writing, it's other thing. The need to move "bin_ztie" function, for example. Who would accept my patch that would do that? (grumbling mode ended). And implementation difficulties, first of all. Two modules, one has to detect the other one, route call. Hm, this seems now quite easy, but when writing I saw the difficulties. And what arises from that is mkfs-like design. Hands up who called mkfs.ext2 or .ext4 instead of mkfs wrapper. And, who would accept Oracle DBM module into Zsh? It's a nice DBM, free, unix-like (chances it nicely fits just like GDBM, has its gdbm_fdesc() call). Then, this database has it's own customs, for example access methods: btree, heap. Chances for new special parameters and stuff. > In keeping with that, there shouldn't be commands named "zgdbm..." and > there should be no "zgdbm_tied" parameter -- there should instead be a > single array or hash in the (so far nonexistent) base module to return > all the ztie'd names, and if there are other ops needed than z(un)tie > then something like "zdb path ...", "zdb clear ...", etc. to give a > consistent front to whatever backend has been passed to "ztie -d". > (Or option names/letters instead of subcommands, to be more consistent > with most builtins, but you get the idea.) Abstraction can go together with specific parts. A general ztie can be maybe accomplished, but having "zgdbm_tied" is like a functionality: "Someone passed me (P flag) a parameter, I can check if it's gdbm frontend via zgdbm_tied array". Then we would have "zodbm_tied" for Oracle DBM. Specific things at first level of use cases, then abstract ones for possible many use cases. > This may mean that there are some ops that either we don't provide for > any backend because they're too specific to one, or that fail for some > backends. E.g. "zdb path" doesn't make sense for a network-connected > database; maybe it returns a URL in that case, or we should consider > that the -f FILENAME argument of ztie is not sufficiently general, and > "zdb path" would just return back that argument of ztie. Ok, such strive for abstraction is a good thing. One has to be in good position in Zsh world to work on that comfortably, think freely about Oracle DBM and its traits, then he can have success in discovering symmetrical API. > None of this means we need to rush off to implement zsh/db/sqlite or > something, but we should be thinking about it when extending db/gdbm. I examined SQLite. There's a no-go – it doesn't return file descriptors for database file and journal file. This is handy example now – maintainers said: "file descriptors work on Unix, it would be not good abstraction to return them". Let's not do it, because it's not abstract. Zsh is purely Unix world, file descriptor would be "fully abstract", but no.. Here is full discussion: http://sqlite.1065341.n5.nabble.com/how-to-get-file-handle-from-sqlite3-object-td28466.html Then, as FDT_INTERNAL nor FDT_MODULE doesn't matter, SQLite might be still possible. I looked at it, to do: result=sqlarray["SELECT telephone FROM contacts WHERE id=1"] And require single column to be selected, or concatenate columns, then also append after \n when "...id=1 OR id=2". This doesn't match DBM usage, though. -- Sebastian Gniazdowski psprint2@fastmail.com ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Gdbm module 2017-02-21 19:52 ` Sebastian Gniazdowski @ 2017-02-22 4:41 ` Bart Schaefer 2017-02-22 9:10 ` Sebastian Gniazdowski 0 siblings, 1 reply; 6+ messages in thread From: Bart Schaefer @ 2017-02-22 4:41 UTC (permalink / raw) To: zsh-workers On Feb 21, 11:52am, Sebastian Gniazdowski wrote: } } [...] who would accept Oracle DBM module into Zsh? Depends entirely on the licensing Oracle specifies. } I looked at [sqlite], to do: } } result=sqlarray["SELECT telephone FROM contacts WHERE id=1"] } } And require single column to be selected, or concatenate columns I would not suggest attempting to make a full SQL interface from the tied hash to the database. GDBM is effectively one table per file with a single unique key column, so the only thing you can do is "SELECT * FROM ...". For a SQL-type database you'd probably want to do something like ztie -d db/sqlite -Q "SELECT ..." ... queryresult and define queryresult in a manner similar to the way Perl DBI does with fetchall_hashref. This theoretical -Q option could be extended to GDBM as a way to filter the set of keys fetched so the hash does not have to load the entire dbmfile. However, at first we might prefer to restrict things to flatter single-table access. Also note I'm only trying to offer some guidelines here; I've seen no evidence that this is needed or more than academically interesting. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Gdbm module 2017-02-22 4:41 ` Bart Schaefer @ 2017-02-22 9:10 ` Sebastian Gniazdowski 2017-02-22 15:44 ` Bart Schaefer 0 siblings, 1 reply; 6+ messages in thread From: Sebastian Gniazdowski @ 2017-02-22 9:10 UTC (permalink / raw) To: zsh-workers On Tue, Feb 21, 2017, at 08:41 PM, Bart Schaefer wrote: > Depends entirely on the licensing Oracle specifies. It's GNU Affero General Public License. Apparently it's like GPL, but aimed at resolving issues with server software "distributed" (that's the point) over network. Just built it, configure runs fast, compilation is long (2:35) but code is C. > ztie -d db/sqlite -Q "SELECT ..." ... queryresult > > and define queryresult in a manner similar to the way Perl DBI does > with fetchall_hashref. Interesting, it assumes single row to be selected in Zsh conditions, as far I understand. It is towards "tie", close or at conforming ztie. > This theoretical -Q option could be extended to GDBM as a way to > filter the set of keys fetched so the hash does not have to load the > entire dbmfile. However, at first we might prefer to restrict things > to flatter single-table access. If a way of specifying keys would have been developed (in above it's for filtering keys) it could be also used to stick ~PM_UPTODATE to that keys. This might somehow translate to SQL. Interesting. Not that I want to patch my up-to-date solution, it allocates via zalloc() and hashtable is always in good shape, previous code was doing hcallloc() and returning internal GDBM string pointers. > Also note I'm only trying to offer some guidelines here; I've seen no > evidence that this is needed or more than academically interesting. I've found use case. Turns out managing plugins via zshrc is rather a pain. If I want to disable single plugin for a while, then `vim ~/.zshrc`, search, comment – it's much tiring. I will provide TUI configurator for Zplugin. Each plugin will have .gdbm file. For TUI I will use my 70% finished Zcurses UI library: https://asciinema.org/a/aner2l63zm6bhmie3nxy3jkc7 It's based on (z) flag :) Having a text-box will mean "zcurses string win ${(q)text}", i.e. quoting, but that's shellish. Bottom line is – it might be possible to map some of Zshrc to gdbm file, to be able to avoid whole `vim ~/.zshrc`, search, etc. cycle. Provided that the cycle is really tiring. So developing DBM modules would be boosted. I would go for writing new module now, to hold ztie, zuntie, ... and route to db_gdbm, but I'm busy with the ZUI lib. Maybe there's time before next release. Good that gdbm is a gem, but Oracle dbm is also very interesting, then there's the "non-existent" Berkley DB (it's probably installed on FreeBSD, a ghost software if not available via tarball with configure, I couldn't find it). -- Sebastian Gniazdowski psprint2@fastmail.com ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Gdbm module 2017-02-22 9:10 ` Sebastian Gniazdowski @ 2017-02-22 15:44 ` Bart Schaefer 2017-02-23 8:19 ` Sebastian Gniazdowski 0 siblings, 1 reply; 6+ messages in thread From: Bart Schaefer @ 2017-02-22 15:44 UTC (permalink / raw) To: zsh-workers On Feb 22, 1:10am, Sebastian Gniazdowski wrote: } } On Tue, Feb 21, 2017, at 08:41 PM, Bart Schaefer wrote: } > ztie -d db/sqlite -Q "SELECT ..." ... queryresult } > } > and define queryresult in a manner similar to the way Perl DBI does } > with fetchall_hashref. } } Interesting, it assumes single row to be selected in Zsh conditions, as } far I understand. Either that, or the keys would be the unique key for each row and the values would be either a single column, or encoded somehow so that they could be split into fields. We are somewhat limited by lacking a mechanism for nested hashrefs, but consider the way "typeset -T" can specify the format of array <-> scalar joining. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Gdbm module 2017-02-22 15:44 ` Bart Schaefer @ 2017-02-23 8:19 ` Sebastian Gniazdowski 0 siblings, 0 replies; 6+ messages in thread From: Sebastian Gniazdowski @ 2017-02-23 8:19 UTC (permalink / raw) To: zsh-workers On Wed, Feb 22, 2017, at 07:44 AM, Bart Schaefer wrote: > We are somewhat limited by lacking > a mechanism for nested hashrefs, but consider the way "typeset -T" can > specify the format of array <-> scalar joining. I plan to write a module that will just blindly put PM_HASH instead of PM_HASHELEM into PM_HASH. See what will happen. The code handling value retrieval is sophisticated, maybe even blind to structure, just follows its PM_HASH etc. detection, I'm counting on that. BTW. Retrieving from PM_SCALAR means double getter call. Saw that when working on db_gdbm/zgdbm. I'm sure there's no error on my side, I've even rolled out complete debugger walk. This could be an optimization, to change/fix that. -- Sebastian Gniazdowski psprint2@fastmail.com ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2017-02-23 8:19 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <CAKc7PVAz=zfrFapifcJudnMqgscNudC5S4S6ym7AWfFFORoROQ@mail.gmail.com> 2017-02-21 17:25 ` Gdbm module Bart Schaefer 2017-02-21 19:52 ` Sebastian Gniazdowski 2017-02-22 4:41 ` Bart Schaefer 2017-02-22 9:10 ` Sebastian Gniazdowski 2017-02-22 15:44 ` Bart Schaefer 2017-02-23 8:19 ` Sebastian Gniazdowski
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/zsh/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).