Re: [Caml-list] stl?

[Pal-Kristian Engstad <pal_engstad@naughtydog.com>]

Jon Harrop wrote:
> On Wednesday 04 March 2009 23:18:21 Pal-Kristian Engstad wrote:
>   
>> Sure -- those are probably not jobs that require performance, nor have
>> resource constraints.
>>     
>
> I do not believe that C++ is significantly faster or better at handling 
> resources than higher-level languages.
>   
Have you ever tried to conform to a specific memory layout? We are often 
talking directly to hardware, and in those cases it is a prerequisite to 
be able to produce data that is in the exact format prescribed. Often 
these things are, put an 17-bit ID followed by a 3-bit CODE followed by 
a 12-bit LENGTH field, after which follows LENGTH items each of size 
that is some-function-of CODE.

This is usually not a problem when a small part of your data needs to be 
described this way, but when a large portion of your data needs this 
formatting, you can see that OCaml or Haskell records simply doesn't 
work very well.

>> I did say most. By the way, XNA is a toy. A good toy, but a toy,
>> nonetheless.
>>     
>
> Note the irony that games are toys. :-)
>   
For the consumer, yes, games are toys. Making games (as well as toys) is 
quite a different story. We're talking multi-million dollar projects here.
>>>>     * Most high-level languages can not support multiple forms of data
>>>>       allocations. Some applications need a range of allocation
>>>>       strategies, ranging from completely automatic (garbage collection)
>>>>       to completely manual.
>>>>         
>>> C++ cannot provide efficient automatic GC.
>>>       
>> That's not true. We run GC on all of our game tasks. It's "manual"-ish,
>> but doable.
>>     
>
> If it is "manual-ish" then it is not automatic!
>   
It is automatic in the sense that it garbage collects automatically at a 
specific time in the frame. It is manual in the sense that you have to 
annotate pointers and other reference like things (e.g. indexes).

>>>>     * Most high-level environments do not allow for fine-grained control
>>>>       of computing resources, e.g. soft real-time guarantees.
>>>>         
>>> Many high-level languages make it easier to satisfy soft
>>> real-time "guarantees", e.g. incremental collection vs destructor
>>> avalanches.
>>>       
>> Call me cynical, but I simply don't buy it.
>>     
>
> I found that when porting Smoke from C++ to OCaml. The worst case performance 
> (which was the problem) got 5x faster in OCaml because the GC did the 
> incremental work that I never managed to get my STL allocators to do 
> effectively. I realised I was just Greenspunning what modern languages 
> already had and that prompted me to drop C++.
>   
It is fairly rare for us to use STL (at least for the run-time portion 
of a game), probably for the reason that you mention. We tend to make 
algorithms and data-structures targeted for the use case.

>> Indeed. But then there are target specific control registers, timers,
>> etc. etc. Usually, these are not supported well.
>>     
>
> So C++ has legacy support for them but they change as hardware evolves and 
> there is no reason why VMs cannot also support them.
>   
True. But do they? Usually not. It is forgotten, deemed a non-important 
thing. The thing is, when you /need/ a hardware specific feature, there 
is usually no out. That was what I was trying to address.

>> Well, first of all - something that doesn't suck performance wise. And
>> it is essential that it works on non-Intel platforms. F# is indeed
>> promising, but again - I would not use it for performance critical code
>> - which is about 30-50% of a game's code base.
>>     
>
> Those are quite tame requirements, IMHO. I'd recommend Cilk.
>   
Cilk supports programming multi-threaded applications on shared-memory 
multiprocessors. That doesn't seem to be applicable to the CELL/SPU 
architecture, for instance. However, I will investigate it further.
>>>> Of course, you can always say that you can use the foreign function
>>>> interface, but then you lose inlining and speed.
>>>>         
>>> The same is true of C/C++. You can get much better performance from
>>> assembler but calling assembler from C or C++ not only costs inlining and
>>> speed but even functionality because you have an ABI to conform to.
>>>       
>> This is not true. Pretty much all C++ compilers have both intrinsic and
>> inline assembly support.
>>     
>
> Ok but that is not specific to C++.
>   
Just another thing that language developers "forget" on the way.

>>>> More importantly, you end up with a project with several different
>>>> languages. That is generally a very bad idea.
>>>>         
>>> A common language run-time is the right solution, not C/C++.
>>>       
>> That is exactly my point. It needs to be *one* language that can cover
>> the broad base from non-performance critical AI code to performance
>> critical culling, animation and physics code.
>>     
>
> A common intermediate representation shared between different front-end 
> languages would suffice.
>   
Are you talking about JIT? Unfortunately, for most consoles you are not 
allowed to write to code-pages, which precludes JIT. Everything must be 
pre-compiled to assembly.

>> But the sad fact is that 
>> there is no competitor to C++. Mind you - I *want* to have something
>> else - it is just not feasible.
>>     
>
> I really don't see why. For example, surely OCaml+LLVM beats C++ in every way 
> that you have described.
>   
LLVM is very interesting indeed, and would be my preferred back-end.
> Moreover, something like my HLVM, which is specifically designed for 
> high-performance computing, should make that vastly easier than C++. It even 
> supports features like optional GC because my GC is written in my IR (and I 
> don't want to GC my GC ;-).
>   
Nice... :-) When will you release your first version?

PKE.

-- 
Pål-Kristian Engstad (engstad@naughtydog.com), 
Lead Graphics & Engine Programmer,
Naughty Dog, Inc., 1601 Cloverfield Blvd, 6000 North,
Santa Monica, CA 90404, USA. Ph.: (310) 633-9112.

"Emacs would be a far better OS if it was shipped with 
 a halfway-decent text editor." -- Slashdot, Dec 13. 2005.