Re: [Edbrowse-dev] Edbrowse recognizing site as binary data

[Sebastian Humenda <shumenda@gmx.de>]

[-- Attachment #1: Type: text/plain, Size: 1288 bytes --]

Hi,

Karl Dahlke schrieb am 22.01.2016,  9:30 -0500:
>> edbrowse https://portal.slm.tu-dresden.de
>> cannot browse a binary file
>> I presume that's a bug? Is there anything I can do to help resolving this issue?
>
>Not a bug technically, it's a feature not yet implemented.
>Save the data to a file and run /bin/file on it and get this
>
>HTML document, Little-endian UTF-16 Unicode text
Ah right.

>The problem is utf-16, which edbrowse does not recognize.
>This has been discussed on the group.
>It is very rare on the internet and becoming rarer still,
This is partially true. Especially for Asian languages, UTF-16 is a better
choice than UTF8. Not true for the above site though.
Anyway, there's
http://llvm.org/svn/llvm-project/llvm/trunk/include/llvm/Support/ConvertUTF.h.
That should provide a conversion function for both UTF-32->UTF-8 and
UTF-16->UTF-8 (together with the appropriate C file, of course). Wouldn't it be
easy to just detect UTF-16 and convert it to UTF-8 before doing anything else?

Cheers
Sebastian
-- 
Web: http://www.crustulus.de (English|Deutsch)  | Blog: http://www.crustulus.de/blog
FreeDict: Free multilingual dictionaries - http://www.freedict.org
Freies Latein-Deutsch-Wörterbuch: http://www.crustulus.de/freedict.de.html

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]