From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx.sdf.org ([205.166.94.20]) by ewsd; Tue Feb 4 18:45:35 EST 2020 Received: from rustbucket.hsd1.il.comcast.net. (c-98-212-152-230.hsd1.il.comcast.net [98.212.152.230]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id 014NjSct028062 (using TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128 bits) verified NO); Tue, 4 Feb 2020 23:45:29 GMT Message-ID: <955870EE382B37038C6DC9E0099709AD@sdf.org> To: kokamoto@hera.eonet.ne.jp, 9front@9front.org Subject: Re: [9front] Netsurf 3.9 for Plan 9 (work in progress) From: Kyle Nusbaum Date: Tue, 4 Feb 2020 17:57:14 -0600 In-Reply-To: <781A1B3FA53B0CDCD43A0D5D8F059442@hera.eonet.ne.jp> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="upas-mblegtlygszqgnhdpiuvdqpyxe" List-ID: <9front.9front.org> List-Help: X-Glyph: ➈ X-Bullshit: secure property-scale dependency full-stack-aware controller This is a multi-part message in MIME format. --upas-mblegtlygszqgnhdpiuvdqpyxe Content-Disposition: inline Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit Yes! I've just applied my changes and have it compiling and loading web pages. I've just taken the git/diff and dumped it. (That's attached) The code still contains debugging fprintf and may contain leaks, etc. It's not done, but it's what I have at the moment. It's kind of a hack, as it hijacks the llcache mechanism, but the higher-level fetcher interface isn't really compatible with webfs. I also have changes that implement correct scrolling and resizing. I'll try to get those into patch format when I get a chance. Hopefully some time this week. --upas-mblegtlygszqgnhdpiuvdqpyxe Content-Type: message/rfc822 Content-Disposition: inline Received: from ewsd.inri.net (ewsd.inri.net [107.191.116.128]) by mx.sdf.org (8.15.2/8.14.5) with ESMTP id 014NeepW009644; Tue, 4 Feb 2020 23:40:40 GMT Received: from cmo-0002.xspmail.jp ([202.238.198.173]) by ewsd; Tue Feb 4 18:40:27 EST 2020 Received: from cmr-0000.xspmail.jp ([202.238.198.117]) by cmo with ESMTP id z7kiiI70bcCQrz7nqiakA5; Wed, 05 Feb 2020 08:40:22 +0900 Received: from pi3L.jitaku.localdomain ([112.71.204.71]) by cmr with ESMTPA id z7nqiHxlW3ObDz7nqimG1P; Wed, 05 Feb 2020 08:40:22 +0900 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hera.eonet.ne.jp; s=x01; t=1580859622; i=kokamoto@hera.eonet.ne.jp; bh=1Zy1igPFn6A5WkwtYrDdP9+gBzW3scvYYTJNvrMZQRw=; h=To:Subject:Date:From:In-Reply-To; b=dtswguY1PiuHeNcSgefLmp+48JyRRwFh0H4JHhOeNcfD4iM5WsL+VNY5FNYHu1D3S U92q5wC/pXgGFNI67/Y+s3npojZyiTcOhbMkZGxv6ZsBfmdiYp+iFkjc1W8sNzVTe3 wBt2s4AEvpGPVYnhCnDg4RkF814YOsArlV9tVmgKS1ROsYwiDRUxeTq01SPmb49ZBO kt+ycMiGN883rLIQgduOzB5RAo6elxZHwOWjOVOJVZeRiazn/gc/8C4r+6UcK+dFc5 5KKH0GofOsKQNBWuAcC9h06nA4MiTuzgtYKvA/5Gj+bRPd6oOQRNh0B3yaPNdmWCza M/Xtu6Gsfdiow== Message-ID: <781A1B3FA53B0CDCD43A0D5D8F059442@hera.eonet.ne.jp> To: 9front@9front.org Date: Wed, 5 Feb 2020 08:40:21 +0900 From: kokamoto@hera.eonet.ne.jp In-Reply-To: 2C561A1243D428B079F8219518FB226C@sdf.org MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" List-ID: <9front.9front.org> List-Help: X-Glyph: ➈ X-Bullshit: encrypted self-healing full-stack configuration injection component Subject: Re: [9front] Netsurf 3.9 for Plan 9 (work in progress) Reply-To: 9front@9front.org Precedence: bulk Content-Transfer-Encoding: 7bit > That works for me. I've not sumbitted patches this way before so correct me if my diffs are not usable. So, are you going to post the diff here? Kenji --upas-mblegtlygszqgnhdpiuvdqpyxe Content-Disposition: attachment; filename=webfs.patch Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit --- /mnt/git/branch/heads/plan9/tree/content/llcache.c Sun Feb 2 19:28:50 2020 +++ content/llcache.c Tue Feb 4 17:33:22 2020 @@ -52,6 +52,7 @@ #include "content/fetch.h" #include "content/backing_store.h" #include "content/urldb.h" +#include "content/webfs.h" /** * State of a low-level cache object fetch. @@ -79,6 +80,8 @@ struct llcache_handle { llcache_fetch_state state; /**< Last known state of object fetch */ size_t bytes; /**< Last reported byte count */ + + webfs_handle *wh; /**< Holds a webfs handle if this is a webfs request. */ }; /** @@ -3326,6 +3329,8 @@ void llcache_clean(bool purge) NSLOG(llcache, DEBUG, "Attempting cache clean"); + webfs_clean(purge); + /* If the cache is being purged set the size limit to zero. */ if (purge) { limit = 0; @@ -3490,6 +3495,11 @@ void llcache_clean(bool purge) nserror llcache_initialise(const struct llcache_parameters *prm) { + nserror err = webfs_initialise(prm); + if(err != NSERROR_OK) { + return err; + } + llcache = calloc(1, sizeof(struct llcache_s)); if (llcache == NULL) { return NSERROR_NOMEM; @@ -3520,6 +3530,8 @@ void llcache_finalise(void) llcache_object *object, *next; uint64_t total_bandwidth = 0; /* total bandwidth */ + webfs_finalise(); + /* Clean uncached objects */ for (object = llcache->uncached_objects; object != NULL; object = next) { llcache_object_user *user, *next_user; @@ -3632,14 +3644,17 @@ nserror llcache_handle_retrieve(nsurl *u nsurl *hsts_url; bool hsts_in_use; + fprintf(stderr, "[DBG]: FETCHING URL: %s\n", nsurl_access(url)); + /* Perform HSTS transform */ error = llcache_hsts_transform_url(url, &hsts_url, &hsts_in_use); if (error != NSERROR_OK) { return error; } + char *scheme = lwc_string_data(nsurl_get_component(hsts_url, NSURL_SCHEME)); /* Can we fetch this URL at all? */ - if (fetch_can_fetch(hsts_url) == false) { + if (fetch_can_fetch(hsts_url) == false && (strcmp("http", scheme) != 0 && strcmp("https", scheme) != 0)) { nsurl_unref(hsts_url); return NSERROR_NO_FETCH_HANDLER; } @@ -3651,6 +3666,17 @@ nserror llcache_handle_retrieve(nsurl *u return error; } + if(strcmp("http", scheme) == 0 || strcmp("https", scheme) == 0) { + // THIS IS A WEBFS REQUEST! + nserror err = webfs_handle_retrieve(hsts_url, flags, referer, post, cb, pw, user->handle, &user->handle->wh); + if(err != NSERROR_OK) { + nsurl_unref(hsts_url); + return error; + } + *result = user->handle; + return err; + } + /* Retrieve a suitable object from the cache, * creating a new one if needed. */ error = llcache_object_retrieve(hsts_url, flags, referer, post, 0, @@ -3682,6 +3708,10 @@ nserror llcache_handle_change_callback(l handle->cb = cb; handle->pw = pw; + if(handle->wh != NULL) { + return webfs_handle_change_callback(handle->wh, cb, pw); + } + return NSERROR_OK; } @@ -3689,6 +3719,9 @@ nserror llcache_handle_change_callback(l /* Exported interface documented in content/llcache.h */ nserror llcache_handle_release(llcache_handle *handle) { + if(handle->wh != NULL) { + return webfs_handle_release(handle->wh); + } nserror error = NSERROR_OK; llcache_object *object = handle->object; llcache_object_user *user = llcache_object_find_user(handle); @@ -3723,12 +3756,20 @@ nserror llcache_handle_clone(llcache_han *result = newuser->handle; } + if(handle->wh != NULL) { + return webfs_handle_clone(newuser->handle, handle->wh, &newuser->handle->wh); + } + return error; } /* See llcache.h for documentation */ nserror llcache_handle_abort(llcache_handle *handle) { + if(handle->wh != NULL) { + return webfs_handle_abort(handle->wh); + } + llcache_object_user *user = llcache_object_find_user(handle); llcache_object *object = handle->object, *newobject; nserror error = NSERROR_OK; @@ -3791,6 +3832,10 @@ nserror llcache_handle_abort(llcache_han /* See llcache.h for documentation */ nserror llcache_handle_force_stream(llcache_handle *handle) { + if(handle->wh != NULL) { + return webfs_handle_force_stream(handle->wh); + } + llcache_object_user *user = llcache_object_find_user(handle); llcache_object *object = handle->object; @@ -3813,6 +3858,10 @@ nserror llcache_handle_force_stream(llca /* See llcache.h for documentation */ nserror llcache_handle_invalidate_cache_data(llcache_handle *handle) { + if(handle->wh != NULL) { + webfs_handle_invalidate_cache_data(handle->wh); + } + if ((handle->object != NULL) && (handle->object->fetch.fetch == NULL) && (handle->object->cache.no_cache == LLCACHE_VALIDATE_FRESH)) { @@ -3826,6 +3875,9 @@ nserror llcache_handle_invalidate_cache_ /* See llcache.h for documentation */ nsurl *llcache_handle_get_url(const llcache_handle *handle) { + if(handle->wh != NULL) { + return webfs_handle_get_url(handle->wh); + } return handle->object != NULL ? handle->object->url : NULL; } @@ -3833,6 +3885,10 @@ nsurl *llcache_handle_get_url(const llca const uint8_t *llcache_handle_get_source_data(const llcache_handle *handle, size_t *size) { + if(handle->wh != NULL) { + return webfs_handle_get_source_data(handle->wh, size); + } + *size = handle->object != NULL ? handle->object->source_len : 0; return handle->object != NULL ? handle->object->source_data : NULL; @@ -3842,6 +3898,10 @@ const uint8_t *llcache_handle_get_source const char *llcache_handle_get_header(const llcache_handle *handle, const char *key) { + if(handle->wh != NULL) { + return webfs_handle_get_header(handle->wh, key); + } + const llcache_object *object = handle->object; size_t i; @@ -3861,5 +3921,14 @@ const char *llcache_handle_get_header(co bool llcache_handle_references_same_object(const llcache_handle *a, const llcache_handle *b) { + if(a->wh != NULL) { + if(b->wh != NULL) { + return webfs_handle_references_same_object(a->wh, b->wh); + } + return false; + } else if(b->wh != NULL) { + return false; + } + return a->object == b->object; } --- /dev/null Wed Jan 22 09:00:06 2020 +++ content/webfs.c Tue Feb 4 17:39:44 2020 @@ -0,0 +1,604 @@ +#include +#include +#include +#include +#include +#include +#include +#include + + +#include "utils/nsurl.h" +#include "content/llcache.h" +#include "netsurf/misc.h" +#include "desktop/gui_internal.h" +#include "libwapcaplet/libwapcaplet.h" +#include "webfs.h" + +// TODO: Condense these. There's no reason we need to divide up the work this much. +typedef enum { + DATA_STATE_UNSTARTED, + DATA_STATE_CLONED, + DATA_STATE_REQUESTED, + DATA_STATE_DATA_READY, + DATA_STATE_DONE, + DATA_STATE_ERROR, +} data_state; + +typedef enum { + HANDLE_STATE_START, // Have not sent LLCACHE_EVENT_HAD_HEADERS yet. + HANDLE_STATE_DATA, // In process of sending LLCACHE_EVENT_HAD_DATA. + HANDLE_STATE_DONE, // We are finished. Either by LLCACHE_EVENT_DONE or LLCACHE_EVENT_ERROR. +} handle_state; + +struct webfs_data { + data_state state; + char *err; + char *urls; + char *pdat; + + int ctlfd; // fd for /mnt/web/N/ctl + int bodyfd; // fd for /mnt/web/N/body + char *webdir; + + char *data; + int datalen; + + int handle_refcount; +}; + +struct webfs_handle { + handle_state state; + llcache_handle_callback cb; + void *pw; + bool aborted; + int sendindex; + + struct webfs_data *data; + + llcache_handle *handle; + + struct webfs_handle *prev; + struct webfs_handle *next; +}; + +struct webfs_handle *handle_head; +bool scheduled; +void update_webfs(void *ignored); + +void add_node(struct webfs_handle *n) { + n->prev = NULL; + n->next = handle_head; + if(handle_head) { + handle_head->prev = n; + } + handle_head = n; + if(!scheduled) { + scheduled=true; + guit->misc->schedule(10, update_webfs, NULL); + } +} + +void remove_node(struct webfs_handle *n) { + if(n->prev) { + n->prev->next = n->next; + } + if(n->next) { + n->next->prev = n->prev; + } + if(handle_head == n) { + handle_head = n->next; + } + n->prev = NULL; + n->next = NULL; +} + +void webfs_send_data(llcache_handle *handle, webfs_handle *wh) { + int tosend = 1024; + if(wh->data->datalen - wh->sendindex < tosend) { + tosend = wh->data->datalen - wh->sendindex; + if(tosend == 0) return; + } + llcache_event ev; + ev.type = LLCACHE_EVENT_HAD_DATA; + ev.data.data.buf = (const unsigned char *)wh->data->data + wh->sendindex; + ev.data.data.len = tosend; + + nserror err = wh->cb(handle, &ev, wh->pw); + if(err == NSERROR_NEED_DATA) { + return; + } + else if (err != NSERROR_OK) { + // TODO: proper error handling + fprintf(stderr, "[DBG]: UNKNOWN ERROR!\n"); + return; + } + else { + wh->sendindex += tosend; + } +} + +/** + * Retrieve the post-redirect URL of a low-level cache object + * + * \param handle Handle to retrieve URL from + * \return Post-redirect URL of cache object + */ +nsurl *webfs_handle_get_url(const webfs_handle *wh) { + nsurl *url; + nserror err = nsurl_create(wh->data->urls, &url); + if(err != NSERROR_OK) { + return NULL; + } + return url; +} + +/** + * Change the callback associated with a low-level cache handle + * + * \param handle Handle to change callback of + * \param cb New callback + * \param pw Client data for new callback + * \return NSERROR_OK on success, appropriate error otherwise + */ +nserror webfs_handle_change_callback(webfs_handle *wh, + llcache_handle_callback cb, void *pw) { + wh->cb = cb; + wh->pw = pw; + return NSERROR_OK; +} + +/** + * Abort a low-level fetch, informing all users of this action. + * + * \param handle Handle to abort + * \return NSERROR_OK on success, appropriate error otherwise + */ +nserror webfs_handle_abort(webfs_handle *wh) { + wh->aborted = true; + return NSERROR_OK; +} + +/** + * Retrieve source data of a low-level cache object + * + * \param handle Handle to retrieve source data from + * \param size Pointer to location to receive byte length of data + * \return Pointer to source data + */ +const uint8_t *webfs_handle_get_source_data(const webfs_handle *wh, + size_t *size) { + *size = wh->data->datalen; + return (const uint8_t *)wh->data->data; + return NULL; +} + +void webfs_data_release(struct webfs_data *d) { + if(d->urls != NULL) { + free(d->urls); + } + if(d->pdat != NULL) { + free(d->pdat); + } + if(d->ctlfd != -1) { + close(d->ctlfd); + d->ctlfd = -1; + } + if(d->bodyfd != -1) { + close(d->bodyfd); + d->bodyfd = -1; + } + if(d->webdir != NULL) { + free(d->webdir); + } + if(d->data != NULL) { + free(d->data); + } + free(d); +} + +/** + * Release a low-level cache handle + * + * \param handle Handle to release + * \return NSERROR_OK on success, appropriate error otherwise + */ +nserror webfs_handle_release(webfs_handle *wh) { + wh->aborted=true; + wh->data->handle_refcount-=1; + if(wh->data->handle_refcount <= 0) { + webfs_data_release(wh->data); + wh->data = NULL; + } + remove_node(wh); + free(wh); + return NSERROR_OK; +} + +/** + * Invalidate cache data for a low-level cache object + * + * \param handle Handle to invalidate + * \return NSERROR_OK on success, appropriate error otherwise + */ +nserror webfs_handle_invalidate_cache_data( ) { + // TODO + return NSERROR_OK; +} + +/** + * Clone a low-level cache handle, producing a new handle to + * the same fetch/content. + * + * \param handle Handle to clone + * \param result Pointer to location to receive cloned handle + * \return NSERROR_OK on success, appropriate error otherwise + */ +nserror webfs_handle_clone(llcache_handle *handle, webfs_handle *wh, webfs_handle **result) { + // TODO (This might be enough already, but revisit and make sure.) + webfs_handle *cp = malloc(sizeof (webfs_handle)); + *cp=*wh; + cp->handle = handle; + cp->sendindex = 0; + cp->state = HANDLE_STATE_START; + cp->next = NULL; + cp->prev = NULL; + *result = cp; + add_node(cp); + return NSERROR_OK; +} + +char *transform_header_name(char *key) { + // TODO I haven't looked at how webfs does this. + // this is just a guess. Need to actually do this + // right. + char *new = malloc(strlen(key)); + char *newp=new; + for(char *c = key; *c != 0; c++) { + if((*c >='a' && *c <='z') || (*c >='0' && *c <= '9')){ + *newp++ = *c; + } + else if(*c >='A' && *c <= 'Z') { + *newp++ = (*c + 32); + } + } + *newp=0; + return new; +} + +/** + * Retrieve a header value associated with a low-level cache object + * + * \param handle Handle to retrieve header from + * \param key Header name + * \return Header value, or NULL if header does not exist + * + * \todo Make the key an enumeration, to avoid needless string comparisons + * \todo Forcing the client to parse the header value seems wrong. + * Better would be to return the actual value part and an array of + * key-value pairs for any additional parameters. + * \todo Deal with multiple headers of the same key (e.g. Set-Cookie) + */ +const char *webfs_handle_get_header(const webfs_handle *wh, + const char *key) { + // TODO Make this function not terrible. + char *newkey = transform_header_name(key); + char fname[1024]; // TODO: Again, static? Too big? + sprintf(fname, "%s%s", wh->data->webdir, newkey); + + int hfd = open(fname, O_RDONLY); + if(hfd < 0) { + return NULL; + } + + char header_content[4097]; // TODO: Again, really far too big and static. + int n = read(hfd, header_content, 4096); + header_content[n]=0; + char *headerbuf = malloc(n+1); + memcpy(headerbuf, header_content, n+1); + close(hfd); + return headerbuf; +} + +/** + * Cause the low-level cache to attempt to perform cleanup. + * + * No guarantees are made as to whether or not cleanups will take + * place and what, if any, space savings will be made. + * + * \param purge Any objects held in the cache that are safely removable will + * be freed regardless of the configured size limits. + */ +void webfs_clean(bool purge) { + // TODO +} + +/** + * Determine if the same underlying object is referenced by the given handles + * + * \param a First handle + * \param b Second handle + * \return True if handles reference the same object, false otherwise + */ +bool webfs_handle_references_same_object(const webfs_handle *a, + const webfs_handle *b) { + return a->data == b->data; +} + +/** + * Force a low-level cache handle into streaming mode + * + * \param handle Handle to stream + * \return NSERROR_OK on success, appropriate error otherwise + */ +nserror webfs_handle_force_stream(webfs_handle *wh) { + // TODO + return NSERROR_NOT_IMPLEMENTED; +} + +/** + * Initialise the low-level cache + * + * \param parameters cache configuration parameters. + * \return NSERROR_OK on success, appropriate error otherwise. + */ +nserror webfs_initialise(const struct llcache_parameters *parameters) { + handle_head = NULL; + scheduled=false; + return NSERROR_OK; +} + +/** + * Finalise the low-level cache + */ +void webfs_finalise(void) { + // TODO +} + +void start_data(struct webfs_data *d) { + int ctlfd = open("/mnt/web/clone", O_RDWR); + if(ctlfd < 0) { + char *e = strerror(errno); + fprintf(stderr, "[DBG]: Failed to clone webfs. Error: %s\n", e); + d->state = DATA_STATE_ERROR; + d->err = "Failed to clone webfs."; // TODO: report strerror? + return; + } + char webd[10]; // 9 digits is plenty. + memset(webd, 0, 10); + int n = read(ctlfd, webd, 9); + webd[n-1]=0; + + d->webdir = malloc(11 + strlen(webd)); // length of "/mnt/web//" + length of webd + NULL + n = sprintf(d->webdir, "/mnt/web/%s/", webd); + d->state = DATA_STATE_CLONED; + d->ctlfd = ctlfd; +} + +void send_request(struct webfs_data *d) { + + if(d->pdat != NULL) { + fprintf(stderr, "[DBG]: Setting POST.\n"); + int n = write(d->ctlfd, "request POST", 12); + if(n <= 0) { + char *e = strerror(errno); + fprintf(stderr, "[DBG]: [%s] Failed to set POST REQUEST. Error: %s\n", d->urls, e); + d->state = DATA_STATE_ERROR; + d->err = "Failed to clone webfs."; // TODO: report strerror? + return; + } + } + + char urlstr[1024]; // TODO dynamic size? + sprintf(urlstr, "url %s", d->urls); + int n = write(d->ctlfd, urlstr, strlen(urlstr)); + if(n <= 0) { + char *e = strerror(errno); + fprintf(stderr, "[DBG]: [%s] Failed to open URL. Error: %s\n", d->urls, e); + d->state = DATA_STATE_ERROR; + d->err = "Failed to clone webfs."; // TODO: report strerror? + return; + } + + d->state = DATA_STATE_REQUESTED; +} + +void start_body(struct webfs_data *d) { + char bodystr[1024]; + sprintf(bodystr, "%sbody", d->webdir); + int bodyfd = open(bodystr, O_RDONLY); + if(bodyfd < 0) { + char *e = strerror(errno); + fprintf(stderr, "[DBG]: [%s] Failed to read body. Error: %s\n", d->urls, e); + d->state = DATA_STATE_ERROR; + d->err = "Failed to clone webfs."; // TODO: report strerror? + return; + } + close(d->ctlfd); //TODO Can we use this? + d->ctlfd = -1; + d->bodyfd = bodyfd; + d->state = DATA_STATE_DATA_READY; +} + +void read_data(struct webfs_data *d) { + llcache_event ev; + char buf[4097]; + buf[4096] = 0; + + int n = read(d->bodyfd, buf, 4096); + if(n < 0) { + char *e = strerror(errno); + fprintf(stderr, "[DBG]: [%s] Failed TO READ FROM BODY. Error: %s\n", d->urls, e); + d->state = DATA_STATE_ERROR; + return; + } + + if(n == 0) { + d->state = DATA_STATE_DONE; + return; + } + + if(d->data == NULL) { + d->data = malloc(n); + } else { + d->data = realloc(d->data, d->datalen + n); + } + memcpy(d->data + d->datalen, buf, n); + d->datalen+=n; +} + +void update_data(struct webfs_data *d) { + if(d->state == DATA_STATE_ERROR || d->state == DATA_STATE_DONE) { + return; + } + else if(d->state == DATA_STATE_UNSTARTED) { + start_data(d); + } + else if(d->state == DATA_STATE_CLONED) { + send_request(d); + } + else if(d->state == DATA_STATE_REQUESTED) { + start_body(d); + } + else if(d->state = DATA_STATE_DATA_READY) { + read_data(d); + } +} + +bool llcache_progress(void *h) { + struct webfs_handle *wh = h; + llcache_handle *handle = wh->handle; + + llcache_event ev; + + // This handle is done. Do nothing. + if(wh->state == HANDLE_STATE_DONE || wh->aborted) { + return false; + } + + // The data has received an error but we haven't reported it + // through the handle yet. + if(wh->data->state == DATA_STATE_ERROR) { + wh->state = HANDLE_STATE_DONE; + ev.type = LLCACHE_EVENT_ERROR; + ev.data.msg = wh->data->err; + wh->cb(handle, &ev, wh->pw); + return false; + } + + + // Make progress on the underlying data. + update_data(wh->data); + + // Main if/else chain. Starting with HANDLE_STATE_START + if(wh->state == HANDLE_STATE_START) { + if (wh->data->state >= DATA_STATE_DATA_READY) { + // Data is ready. + ev.type = LLCACHE_EVENT_HAD_HEADERS; + wh->cb(handle, &ev, wh->pw); + wh->state = HANDLE_STATE_DATA; + } + } + else if(wh->state == HANDLE_STATE_DATA) { + // We know data is ready. + if(wh->sendindex == wh->data->datalen) { + // We're at the end. Check if the data is still coming. + if(wh->data->state == DATA_STATE_DONE) { + // We sent all the data and no more is coming! + ev.type = LLCACHE_EVENT_DONE; + wh->cb(handle, &ev, wh->pw); + wh->state = HANDLE_STATE_DONE; + return true; + } + // We have no data now, but more is coming. + return true; + } + else { + webfs_send_data(handle, wh); + } + } + return true; +} + +void update_webfs(void *ignored) { + struct webfs_handle *wh = handle_head; + if(wh == NULL) { + // Return early to stop scheduling. + scheduled=false; + return; + } + while(wh != NULL) { + struct webfs_handle *next = wh->next; + if(!llcache_progress(wh)) { + remove_node(wh); + } + wh = next; + } + guit->misc->schedule(10, update_webfs, NULL); +} + + +/** + * Retrieve a handle for a low-level cache object + * + * \param url URL of the object to fetch + * \param flags Object retrieval flags + * \param referer Referring URL, or NULL if none + * \param post POST data, or NULL for a GET request + * \param cb Client callback for events + * \param pw Pointer to client-specific data + * \param result Pointer to location to recieve cache handle + * \return NSERROR_OK on success, appropriate error otherwise + */ +nserror webfs_handle_retrieve(nsurl *url, uint32_t flags, + nsurl *referer, const llcache_post_data *post, + llcache_handle_callback cb, void *pw, llcache_handle *handle, + webfs_handle **result) { + + // We are going to ignore flags for now. + + if(post != NULL) { + // TODO Implement POST data. + if(post->type == LLCACHE_POST_URL_ENCODED) { + fprintf(stderr, "[DBG]: POST DATA: [%s]\n", post->data.urlenc); + } else { + fprintf(stderr, "[DBG]: POST DATA NOT IMPLEMENTED!\n"); + } + return NSERROR_BAD_CONTENT; + } + + char *scheme = lwc_string_data(nsurl_get_component(url, NSURL_SCHEME)); + if(strcmp("http", scheme) != 0 && strcmp("https", scheme) != 0) { + // TODO proper error handling. + fprintf(stderr, "[DBG]: Cannot handle the scheme [%s]\n", scheme); + abort(); + } + + webfs_handle *wh = malloc(sizeof *wh); + wh->state = HANDLE_STATE_START; + wh->cb = cb; + wh->pw = pw; + wh->aborted = false; + wh->sendindex = 0; + wh->handle = handle; + wh->next = NULL; + wh->prev = NULL; + wh->data = malloc(sizeof *wh->data); + + wh->data->state = DATA_STATE_UNSTARTED; + char *url_cstr = nsurl_access(url); + wh->data->urls = malloc(strlen(url_cstr) + 1); + strcpy(wh->data->urls, url_cstr); + wh->data->pdat = NULL; + wh->data->ctlfd = -1; + wh->data->bodyfd = -1; + wh->data->webdir = NULL; + wh->data->data = NULL; + wh->data->datalen = 0; + wh->data->handle_refcount = 1; + add_node(wh); + + *result = wh; + + return NSERROR_OK; +} --- /dev/null Wed Jan 22 09:00:06 2020 +++ content/webfs.h Tue Feb 4 17:21:39 2020 @@ -0,0 +1,29 @@ +#ifndef NETSURF_CONTENT_WEBFS_H_ +#define NETSURF_CONTENT_WEBFS_H_ + +typedef struct webfs_handle webfs_handle; + +nsurl *webfs_handle_get_url(const webfs_handle *wh); +nserror webfs_handle_change_callback(webfs_handle *wh, + llcache_handle_callback cb, void *pw); +nserror webfs_handle_abort(webfs_handle *wh); +const uint8_t *webfs_handle_get_source_data(const webfs_handle *wh, + size_t *size); +nserror webfs_handle_release(webfs_handle *wh); +nserror webfs_handle_invalidate_cache_data(webfs_handle *handle); +nserror webfs_handle_clone(llcache_handle *handle, webfs_handle *wh, webfs_handle **result); +const char *webfs_handle_get_header(const webfs_handle *wh, + const char *key); +void webfs_clean(bool purge); +bool webfs_handle_references_same_object(const webfs_handle *a, + const webfs_handle *b); +nserror webfs_handle_force_stream(webfs_handle *wh); +nserror webfs_initialise(const struct llcache_parameters *parameters); +void webfs_finalise(void); +nserror webfs_handle_retrieve(nsurl *url, uint32_t flags, + nsurl *referer, const llcache_post_data *post, + llcache_handle_callback cb, void *pw, llcache_handle *handle, + webfs_handle **result); + + +#endif --- /mnt/git/branch/heads/plan9/tree/mkfile Sun Feb 2 19:28:50 2020 +++ mkfile Tue Feb 4 17:34:21 2020 @@ -40,6 +40,7 @@ OBJ=\ content/fetch.$O \ content/hlcache.$O \ content/llcache.$O \ + content/webfs.$O \ content/mimesniff.$O \ content/urldb.$O \ content/no_backing_store.$O \ --upas-mblegtlygszqgnhdpiuvdqpyxe--