From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on yquem.inria.fr X-Spam-Level: * X-Spam-Status: No, score=1.0 required=5.0 tests=AWL,SPF_NEUTRAL autolearn=disabled version=3.1.3 X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by yquem.inria.fr (Postfix) with ESMTP id 32BD5BC69 for ; Tue, 16 Oct 2007 04:20:27 +0200 (CEST) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgAAAHO8E0fAXQInh2dsb2JhbACOSQEBAQgKKYEn X-IronPort-AV: E=Sophos;i="4.21,281,1188770400"; d="scan'208";a="4566091" Received: from concorde.inria.fr ([192.93.2.39]) by mail3-smtp-sop.national.inria.fr with ESMTP; 16 Oct 2007 04:20:26 +0200 Received: from mail4-relais-sop.national.inria.fr (mail4-relais-sop.national.inria.fr [192.134.164.105]) by concorde.inria.fr (8.13.6/8.13.6) with ESMTP id l9G2KQjA015063 (version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=OK) for ; Tue, 16 Oct 2007 04:20:26 +0200 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgAAADa8E0dC+Vyri2dsb2JhbACOSQEBAQgEBAkKEQWBKQ X-IronPort-AV: E=Sophos;i="4.21,281,1188770400"; d="scan'208";a="18057155" Received: from ug-out-1314.google.com ([66.249.92.171]) by mail4-smtp-sop.national.inria.fr with ESMTP; 16 Oct 2007 04:20:25 +0200 Received: by ug-out-1314.google.com with SMTP id m3so14459ugc for ; Mon, 15 Oct 2007 19:20:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:received:date:to:subject:message-id:mail-followup-to:references:mime-version:content-type:content-disposition:in-reply-to:user-agent:from; bh=iKRBGJ8cl62HwsJ3y6D77JfQ5AFARdpcxkFy4ny+1VU=; b=dnwX+c3KidosvLpP09gNqAWindQxG/WrCB1bag1YYzzThxlwjvM4zUyydSuMDuO4lbA1ERC1ffwX5P9h7+iBF1POarkIc4mruxrApXjESN3esMf5CCto7+RqJHQVUjMkK1Xi7Rcsl1gYKl1AsZC3VY66FY32FB6Una7EBiJLmRw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:date:to:subject:message-id:mail-followup-to:references:mime-version:content-type:content-disposition:in-reply-to:user-agent:from; b=DEdCJyECs4HsXxirxMpD3NPy0/fNZZ0GdRHWz/flcGdq9wcnLNvYGjV4W/qDIwNIAFmYwqEu9jYW+Hjzcfa0sIVvohDIS3ypYdvkz4gQq6UBahE036GBTZAn6kiBUZmilNce2PcvdTi9oeUOFexcyGyby8oho1m7W1Wyt3CUTOw= Received: by 10.66.250.18 with SMTP id x18mr8550488ugh.1192501221581; Mon, 15 Oct 2007 19:20:21 -0700 (PDT) Received: from localhost ( [83.201.28.205]) by mx.google.com with ESMTPS id z33sm4389760ikz.2007.10.15.19.20.17 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 15 Oct 2007 19:20:19 -0700 (PDT) Received: by localhost (sSMTP sendmail emulation); Tue, 16 Oct 2007 04:21:37 +0200 Date: Tue, 16 Oct 2007 04:21:37 +0200 To: caml-list@inria.fr Subject: Re: [Caml-list] Warning on home-made functions dealing with UTF-8. Message-ID: <20071016022136.GA26814@localhost> Mail-Followup-To: Julien Moutinho , caml-list@inria.fr References: <20071009165520.GA27507@snarc.org> <6f9f8f4a0710091032r3dace80bi4f9b584ae5056675@mail.gmail.com> <20071009195119.GA29263@snarc.org> <875c7e070710091504j203ff78y8703fc5c22086671@mail.gmail.com> <20071011130354.GA7016@snarc.org> <1192110864.6045.12.camel@rosella.wigram> <20071011142141.GA8001@snarc.org> <1192114097.6184.7.camel@rosella.wigram> <20071015203509.GA5212@localhost> <1192492276.22001.27.camel@rosella.wigram> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <1192492276.22001.27.camel@rosella.wigram> User-Agent: Mutt/1.5.16 (2007-06-11) From: Julien Moutinho X-Miltered: at concorde with ID 47141FEA.000 by Joe's j-chkmail (http://j-chkmail . ensmp . fr)! X-Spam: no; 0.00; home-made:01 0200,:01 backslash:01 wrote:01 wrote:01 readable:01 caml-list:01 functions:01 algorithm:01 parse:02 string:02 tue:06 indeed:07 correct:08 99%:90 On Tue, Oct 16, 2007 at 09:51:16AM +1000, skaller wrote: > On Mon, 2007-10-15 at 22:35 +0200, Julien Moutinho wrote: > > Just in case someone would want to use this parse_utf8, > > be aware that depending on the trust you have in your input, > > it may be sorely discouraged to do so. > > Indeed, this code does not check comprehensively for invalid characters. > > That is correct. It is specifically designed NOT to do so. At your own risk, there was no offense. > The last thing you want in 99% of codec use is to abort due > to an error. > > Try switching codecs on Firefox.. do you really want to abort > if you have bad input or the wrong codec? I would say, whatever Firefox does I want it to be a minimun safe. > UTF-8 is primarily used for Unicode which is human readable text. > Errors and faults in the text aren't important most of the time. Unless they are voluntarily put by a malicious assailant. cf. [1] where a backslash is used with the overlong UTF-8 form "\xC1\x9C" instead of "\x5C", fooling IIS' string search algorithm. > It has nothing to do with a 'trusted' source. It has to do with > the fact that the human text is an approximation in the first > place. [1] http://www.securityfocus.com/bid/1806/discuss