Re: [Caml-list] stl?

[Jon Harrop <jon@ffconsultancy.com>]

On Thursday 05 March 2009 02:15:20 Pal-Kristian Engstad wrote:
> Jon Harrop wrote:
> > On Wednesday 04 March 2009 23:18:21 Pal-Kristian Engstad wrote:
> >> Sure -- those are probably not jobs that require performance, nor have
> >> resource constraints.
> >
> > I do not believe that C++ is significantly faster or better at handling
> > resources than higher-level languages.
>
> Have you ever tried to conform to a specific memory layout? We are often
> talking directly to hardware, and in those cases it is a prerequisite to
> be able to produce data that is in the exact format prescribed. Often
> these things are, put an 17-bit ID followed by a 3-bit CODE followed by
> a 12-bit LENGTH field, after which follows LENGTH items each of size
> that is some-function-of CODE.
>
> This is usually not a problem when a small part of your data needs to be
> described this way, but when a large portion of your data needs this
> formatting, you can see that OCaml or Haskell records simply doesn't
> work very well.

I agree with the symptoms but not with C++ as the treatment. Granted you 
cannot write such code directly in OCaml or Haskell but you can generate the 
code using tools like LLVM without losing the benefits of high-level 
programming and I definitely prefer that to writing C++ by hand.

> >> That's not true. We run GC on all of our game tasks. It's "manual"-ish,
> >> but doable.
> >
> > If it is "manual-ish" then it is not automatic!
>
> It is automatic in the sense that it garbage collects automatically at a
> specific time in the frame. It is manual in the sense that you have to
> annotate pointers and other reference like things (e.g. indexes).

Ok, if you're doing the annotations by hand then it is not automatic memory 
management IMHO.

> > I found that when porting Smoke from C++ to OCaml. The worst case
> > performance (which was the problem) got 5x faster in OCaml because the GC
> > did the incremental work that I never managed to get my STL allocators to
> > do effectively. I realised I was just Greenspunning what modern languages
> > already had and that prompted me to drop C++.
>
> It is fairly rare for us to use STL (at least for the run-time portion
> of a game), probably for the reason that you mention. We tend to make
> algorithms and data-structures targeted for the use case.

Yes, I'm not surprised.

> >> Indeed. But then there are target specific control registers, timers,
> >> etc. etc. Usually, these are not supported well.
> >
> > So C++ has legacy support for them but they change as hardware evolves
> > and there is no reason why VMs cannot also support them.
>
> True. But do they? Usually not. It is forgotten, deemed a non-important
> thing. The thing is, when you /need/ a hardware specific feature, there
> is usually no out. That was what I was trying to address.

I see. Yes, definitely sounds like you need an extensible performant 
high-level language implementation.

> >> Well, first of all - something that doesn't suck performance wise. And
> >> it is essential that it works on non-Intel platforms. F# is indeed
> >> promising, but again - I would not use it for performance critical code
> >> - which is about 30-50% of a game's code base.
> >
> > Those are quite tame requirements, IMHO. I'd recommend Cilk.
>
> Cilk supports programming multi-threaded applications on shared-memory
> multiprocessors. That doesn't seem to be applicable to the CELL/SPU
> architecture, for instance. However, I will investigate it further.

I have no idea about CELL/SPU, sorry.

> >> This is not true. Pretty much all C++ compilers have both intrinsic and
> >> inline assembly support.
> >
> > Ok but that is not specific to C++.
>
> Just another thing that language developers "forget" on the way.

I'll keep that in mind...

> >>>> More importantly, you end up with a project with several different
> >>>> languages. That is generally a very bad idea.
> >>>
> >>> A common language run-time is the right solution, not C/C++.
> >>
> >> That is exactly my point. It needs to be *one* language that can cover
> >> the broad base from non-performance critical AI code to performance
> >> critical culling, animation and physics code.
> >
> > A common intermediate representation shared between different front-end
> > languages would suffice.
>
> Are you talking about JIT? Unfortunately, for most consoles you are not
> allowed to write to code-pages, which precludes JIT. Everything must be
> pre-compiled to assembly.

It doesn't need to be a JIT and, actually, HLVM already supports both JIT and 
standalone compilation.

> >> But the sad fact is that
> >> there is no competitor to C++. Mind you - I *want* to have something
> >> else - it is just not feasible.
> >
> > I really don't see why. For example, surely OCaml+LLVM beats C++ in every
> > way that you have described.
>
> LLVM is very interesting indeed, and would be my preferred back-end.

Takes a lot of learning but LLVM is awesome once you've got to grips with it. 
I'm hoping my high-level interface can take a lot of the pain out of using 
it...

> > Moreover, something like my HLVM, which is specifically designed for
> > high-performance computing, should make that vastly easier than C++. It
> > even supports features like optional GC because my GC is written in my IR
> > (and I don't want to GC my GC ;-).
>
> Nice... :-) When will you release your first version?

I'm just writing the GC now and then I'll release a first version.

I'm hoping JLouis can make a MosML front end quickly. ;-)

-- 
Dr Jon Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?e