It's official, foreach is bad (in Unity)

edited in General
This really pains me because I love using foreach. @AngryMoose, on the other hand, will probably rejoice :)

From Gama:

Matthew Hanlon pointed my attention to an unfortunate (yet also very interesting) discrepancy between Microsoft's current C# compiler and the older Mono/C# compiler that Unity uses 'under the hood' to compile your scripts on-the-fly. You probably know that you can use Microsoft Visual Studio to develop and even compile Unity/Mono compatible code. You just drop the respective assembly into the 'Assets' folder. All code is then executed in a Unity/Mono runtime environment. However, results can still differ depending on who compiled the code! foreach loops are just such a case, as I've only now figured out. While both compilers recognize whether a collection's GetEnumerator() returns a struct or a class, the Mono/C# has a bug which 'boxes' (see below, on boxing) a struct-enumerator to create a reference type.
So while it should be ok to foreach, thanks to a bug in Unity's (very ooooold) Mono compiler version it really isn't unless you are compiling (not editing) your code in Visual Studio, and that's a pretty limiting restriction:

So should you avoid foreach loops?

Don't use them in C# code that you allow Unity to compile for you.
Do use them to iterate over the standard generic collections (List<T> etc.) in C# code that you compile yourself with a recent compiler. Visual Studio as well as the free .NET Framework SDK are fine, and I assume (but haven't verified) that the one that comes with the latest versions of Mono and MonoDevelop is fine as well.

Comments

  • How do you 'heart' something more than once?
  • edited
    Well it's disappointing. Sometimes with mono it feels like we are back to 20 years ago... :(

    I am interested, however, to see the measured performance penalty, as opposed to the theoretical "it's bad". There are lots of performance myths flying around in the Unity world.

    Also, there is some limitations with the AOT compiler (used to compile C# code for iOS) that prevents certain list operations to be executed on generic lists which contains value types (for instance, Contains, Remove, etc. crash at runtime). To overcome this, I rolled out my own implementation of IList<...> (in my case, most of the functions delegate to an underlying list; I just provided alternative implementations for the methods that don't work). This is fairly easy to do, and I'd rather use my own list than forgo foreach, especially if you have an implementation at hand already....

    Or you could write an extension method to get the right enumerator:

    foreach(var item in list.SafeEnum) { ... } //or is this risky??

    EditThe article says LINQ does not work on iOS. This is mostly false. Certain things do not work, but that has to do with the underlying collections rather than LINQ itself. Our grid library use LINQ extensively. (We had to do some work to get certain algorithms and methods working, but again, it was the collections and extremely generic code, not LINQ).

    Thanked by 1Fengol
  • Garbage is garbage. You're not going to notice a performance hit using foreach, but it's going to box and generate garbage, how much depends on what it is that you are evaluating in the foreach, how many iterations, how many times it is called, etc. Your performance hit will be when Mono stalls to do a collection, which could be different on every platform... on desktops, you probably wont ever notice a hit. On mobile, you'll probably see a frame with a 10ms+ performance hit on the CPU. If you don't have the ms to spare to stay above 30fps or 60fps, you're players are going to experience a jitter. If you have enough garbage generation going on, you may have a jitter every few seconds, which is going to completely destroy your experience.

    Bottom line, from experience, avoid anything and everything that generates garbage during interactive gameplay sections. Setup time and even pause menu areas are usually fine to go nuts with garbage code if it saves you time. Call a GC.Collect() whenever players wont notice a stutter (ie. when you pause the game, when you change pages in the menu, etc.) to be proactive about garbage build up and minimize future collects.

    The biggest thing to avoid is any sort of String work at runtime... even just calling .toString() to get the player's current score every frame can land you in crazy garbage trouble quickly, so watch you!

    Unity does have a great suite of profilers for you to make use of, especially WRT garbage generate every frame, so if you make sure of them it's ridiculously easy to track down and address those issues when you need to.

    Also, those recent Gamasutra articles on Unity memory management cover a lot of decent practices and are a good staring place if you are having issues with using too much memory or too many garbage collect, but there are tons more tricks than what they cover, so keep your brains open!
  • Yup, I meant those spikes on garbage collection with "performance penalty".

    I think articles such as this one is useful to understand what's going on and to know where to look when things go bad. However, I am a bit skeptical about following such practices in general, mostly because they don't tell how how bad something is compared to other bad things. (I mean, how many programmers still cache Unity's transforms because it is supposedly slow?)

    Doing something extreme to avoid a little garbage (compared to overall garbage) does not seem so useful. (Of course, avoiding foreach is not very extreme). Avoiding something that genertes lots of garbage (compared to overall garbage) is indeed useful.

    But I 100% agree with using profilers; that takes care of it!
  • hermantulleken said:
    I am interested, however, to see the measured performance penalty, as opposed to the theoretical "it's bad". There are lots of performance myths flying around in the Unity world.
    That's one of the reasons I like this series of articles, not only does the author back it up with real tests, the comments have provided some really useful additional information. If you read the article, and more importantly the comments, you'll see that the problem is that the default implementation should be free of garbage (the GetEnumeration methods in System.Collections.Generic return structs, not classes) but it's a compiler error in the old version of Mono that Unity still uses that boxes that returned struct.

    On the amount of garbage this generates, it's not about lots of garbage, it's that every little bit adds up. The additional allocations from boxing may seem tiny, but the problem isn't their size but that they add one more thing the creaky old mono GC has to check. The more items the GC has to check, the longer the pause when it does run.

    So yeah, while I agree you shouldn't do extreme things to avoid GC until you know you have an issue, If there's an easy rule of thumb that will help you avoid garbage generation like "don't use foreach in game-time code", it's worth sticking to it. Yes you could go and generate an alternative implementation, but without testing it explicitly you don't know that you won't have the same problem. And in my experience rolling your own version of something that already in the library but just missing one tiny little feature that you can live without is almost always the wrong choice.

    On strings and garbage, something interesting that was pointed out in the article as well, was that string.Format is also a culprit. Because it boxes any value types that get passed to it. So really, the most garbage-efficient way to build up a string is using a StringBuilder and it's typed .Append methods.

  • Yay! One advantage for me by not using Unity! Just kidding :D
  • When you guys talk about GC.collect, is it literally that, and does that literally make Unity do a memory check and clear out unused garbage data in memory? That's the gist of what I'm understanding from this reading.

    Or shouldn't I worry about this until I'm much more pro? :|
  • Such an unnecessary bug. Hopefully they can update the Mono compiler soon.
  • Tuism said:
    Or shouldn't I worry about this until I'm much more pro?
    Pretty much, but.. if you find you get to a point on one of your games/protos where everything runs smoothly but you get occasional frame hitches, this could be the culprit. Of course as with any performance issue, you should find a way to narrow down what the problem really is before wasting time solving problems you don't actually have.
    Tuism said:
    does that literally make Unity do a memory check and clear out unused garbage data in memory
    As luck would have it, we have a handy video on this by Christopher Myburg on the MGSA youtube channel ;)
    Squidcor said:
    Such an unnecessary bug. Hopefully they can update the Mono compiler soon.
    It sounds very unlikely. Unity have apparently modified the Mono runtime and compiler to suit their needs, and these changes may make an update difficult. By all means vote for this change here though:

    http://feedback.unity3d.com/suggestions/update-built-in-mono-compiler-and-runtime-to-a-newer-version
  • edited
    I put a vote in for the mono compiler update and tweeted the link as well. Thanks!
  • I put 2 votes, I have no idea how the voting works (couldn't find any resources :( ) so Matt advised me not to put all 10 in.
  • Yoh, I must say I had a wave of rage pass me when I read that it's unlikely because of what they did to the compiler... For being in a supposedly "cutting edge industry" (technology-wise) we are always always always behind. It drives me i n s a n e. (Heaven forbid that we should give up mecanim or shuriken or umbra or substances... just so they have some dev time to spend to build us an up-to-date compiler.... //// rant

    Anyways, @mattbenic, what you say is sane. I did read the article with the data he reports, but what is 12 KB per frame? How much does a game typically generate? For me, that is just a number. For all I know a game could generate GBs of garbage per frame. That is what I mean with the utility with these types of articles. Yes, maybe I know it's probably not GBs... but other may not, and then blindly follow the rules of thumb, which in many games simply don't matter.

    But again, I agree with you intellectually, but I think we have slightly different different philosophies. (The idea to not use LINQ makes me want jump into a black hole :P)
  • 12 KB per frame @ 60fps = a garbage collection (and therefore, usually a CPU spike) every 1.42 seconds in the case where Mono collects after 1MB of garbage has been collected (which is often true)... but generational GCs have Dark Magics(tm) behind them and the 'when' of a GC is not always known.
  • edited
    The old mono version of unity uses a naive mark and sweep implementation. Beyond just expected foreach behaviour, I would say the access to a generational garbage collector in more recent versions of mono would be a pretty huge win.

    Basically a generational GC maintains a nursery pool of objects that is much smaller than the entire working memory of the program. This means most of the frame to frame cruft can be quickly GC'd (without stutter) and the long lived objects are deferred to a full blown GC at a later stage. Whilst I've never used a generational GC I imagine this would make the whole class of problems we are talking about here pretty irrelevant.

    Also I think in line with what @hermantulleken is saying, it's always worth asking if something happens per frame or just once. Usually per frame can change any equation drastically. GUI with literal strings declared everywhere equals ALL THE GARBAGE.

  • Feel free to vote for this issue which focuses on the GC as I mentioned above, has more traction and will achieve both aims. (I voted for both anyway):
    http://feedback.unity3d.com/suggestions/scripting-garbage-collection-sg
    Thanked by 1mattbenic
  • edited
    HermanTulleken said:
    But again, I agree with you intellectually, but I think we have slightly different different philosophies. (The idea to not use LINQ makes me want jump into a black hole :P)
    I haven't done any allocation testing on LINQ at all, but my gut (and the MS guys we worked with on the Harvest IIRC) tell me it's a garbage generating mofo because of it's heavy use of anonymous delegates and Count extension.
    As such, while I agree it's awesome, I prefer to only use it in setup and load time-basically places where I expect there to be tons of allocation already. But in those places, hell yeah it's amazing to use :)

    A question for you @HermanTulleken: If you're such a big LINQ user in grids, have you perhaps profiled it for garbage allocation in one of your 30/30 games? I'd be very curious to see the results of that.
    TheFuntastic said:
    Feel free to vote for this issue which focuses on the GC as I mentioned above, has more traction and will achieve both aims. (I voted for both anyway):
    http://feedback.unity3d.com/suggestions/scripting-garbage-collection-sg
    Missed that because I didn't search for garbage, only for compiler and runtime. Doh ;) Voted.
  • HermanTulleken said:
    How much does a game typically generate?
    Actually, it's not that hard to get your typical frame allocation down to zero. If you're handling strings correctly (only updating them when the underlying values change), avoiding anonymous delegates and boxing, and pooling your objects, you should only have allocations happening occasionally (as in your typical frame should be allocation free).
    Unity does unfortunately include an input allocation you can't get rid of (on PC) and I suspect that's because they're using an input call in .Net that's known to allocate unnecessarily-might even be fixed in Unity 4.

    But really, 12k per frame is pretty damned high (as @AngryMoose pointed out, that'll hit a 1 meg limit in under 2 seconds). On The Harvest we were down to zero or single to double digit byte allocations per frame-enough to almost entirely eliminate allocations between menu sessions.
    Thanked by 1Chippit
  • edited
    @mattbenic Although we do use LINQ everywhere, it tends to be mostly in grid setup and grid algorithms, which are typically not called frame by frame. One of the few examples among our games I could think of was Game 7, where we call AStar every frame (if the game were optimised, we would only call it when necessary, but it gives at least a bit of a test case.)

    I did a few quick tests: on my machine a garbage collect (taking about 4.2ms) about after 70 seconds, and there after irregularly at about 200 frames or so. Taking out the AStar algorithm did not seem to make a difference, nor did putting in a foreach loop over the grid in Update... (But of course these are not scientific tests). 4.2 ms is not something I would normally be worried about.

    Also, I see something creates about 120 meshes every second, and destroys them again after 5 frames... (certainly not something we are doing... maybe the profiler renderer? Does that show up in the profiler?) I would not be surprised if that contributes to garbage more than the program language overhead. I will do some more tests when I have some more time, and let you know what I find.
  • Remember though that the pain of performing a GC increases with the size of your program as GC has to traverse your entire working memory. Also going to Mobile is a fun way to insta make that number way bigger and infinitely more painful.
  • edited
    @TheFuntastic Of course :P I never had to replace foreach loops with for loops to solve a performance problem before. In my experience, when you have perfomance problems, something else is so bad, you can get all the performance you need by fixing that instead. But yes, without a real program to profile, it's all moot. (Edit: With performance problems I include GC spikes).

    I did another quick test. I took out everything (including rendering), and just had the foreach and for in Update. The foreach still caused some garbage collects (but much less frequently). For does not at all. As we of course suspect given the article. (It did surprise me however, since I tested it on a grid, not a list, and our grid does have a custom enumerator that does not box values... so something else is going on...) Also, the mesh creation is indeed the profiler (if you show less profilers, it creates fewer meshes... just for interests sake. Also, it seems to have no effect on garbage).

    ((Just a quick aside before anyone thinks our library is a slow garbage generator, which you may well given what I said earlier. We took great pains to optimise it where it matters. For instance, grid elements can be accessed constant time, no matter how complicated your shape or how many elements you have. We _do_ use LINQ and IEnumerables heavily, but this allows you to write fast code that is also neat. For instance, the lazy computation nature of enumerators allow you to say for instance grid.Any(p => (p - q).Magnitude < 3), which stops computation as soon as the first point is found. Another one is grid.SampleRandom(3), which returns three random points (lazily); something that can be very complicated to do with for loops given arbitrary grid shapes. And we have made points structs so that you can generate thousands without worrying about garbage (so you can say (p + q)*2 every frame, even though we had to give up inheritance to do that, and had to use our own version of List internally so that it can compile on iOS. We _do_ care about performance! Sorry for this derailment!))
  • edited
    hermantulleken said:
    and our grid does have a custom enumerator that does not box values
    I think the point in the article was that the enumerator itself gets boxed, not the values from it :)

    Based on what you're describing, I'd love to see a talk from you at a community meet about using linq. I've started using it a bit (in tools and load time code), but it sounds like you've got to know it (and it's performance implications) quite well and could give a good intro to it. It's been a while since we had a more technical talk anyway ;)
  • I'll second that Linq talk!
  • If people are interested in it; I'd love too :)
  • I would love a linq talk. Record it!
  • LINQ is made of so much win and brings functional programming to C# and .NET
  • I've read about performance issues with Linq and upon doing some research (a quick google search 8| ) I came across this which looked useful enough to be worth a share.

    Check the answer not so much the actual question ->

    http://stackoverflow.com/questions/4044400/linq-performance-faq
  • edited
    LINQ and performance is sure interesting, and it is not always clear what is best.

    For instance, using the laziness of LINQ could indeed provide performance benefits in some cases. But it is difficult. You have to design your algorithms a certain way, and indeed may need to design your whole program a certain way. Furthermore, you'd have to have a quite deep understanding of how the LINQ functions are implemented: is Count O(n)? Is Reverse lazy? In many cases you do not even have access to the knowledge without breaking abstraction, and risking your performance going down the drain if one of the implementations change. As is often the case with performance issues, knowledge is a dangerous thing.

    There is also the issue of multiple iteration. The answer gives a filtering example, but what if you need to do a query twice? Is it better to keep the laziness, or forego it so that you only need to iterate once? In general it may be difficult to know. And even if you do know there is again the risk that it will change...

    And then there are all the strange indirect costs. I recently did a test on a very tough problem (by game dev standards, taking some 10 seconds to execute in the version I optimised the most heavily). One of my optimisations was to replace LINQ code with normal array and for loops. It made a huge difference, but it turns out a large part of it came from replacing IEnumerators with concrete types, and not LINQ per se. Unfortunately, I could not determine exactly what was going on; it seems that the Unity profiler does not report frames that take over a certain amount of time to execute o.0!

    The good news is, in a very large number of games, LINQ code makes up such a small part of the execution time that these concerns are pretty much irrelevant.

    My general rule of thumb is this: when your code is so intense that using LINQ makes a difference, it means you are doing very interesting things with your data... Those interesting things are 9/10 times a better place to start looking for optimisation, and very often saves much more than removing LINQ would have. Programming is not about writing the most efficient programs possible, it's about using your time as efficiently as possible to write correct maintainable programs that run fast enough.
  • LINQ also creates loads of garbage, but it can often make your code shorter and easier to understand/maintain.

    As my general rule of thumb I avoid using LINQ in FixedUpdate/Update/LateUpdate methods, use it sparingly in collision-related and other event-like methods (or in Awake and Start methods on objects that are created dynamically), and use it freely in Awake and Start methods that only run when the scene is loaded (but preferably followed by a manual invocation of the garbage collector before the scene starts updating).

    Though the performance cost of very complex LINQ queries can potentially impact the framerate at runtime, as part of scene initialization the impact on loading times is highly unlikely to be perceivable by anyone.
    Thanked by 1TheFuntastic
  • edited
    Coming from an internet start-up background where productivity is king, I grew incredibly weary of purposely avoiding Linq and actually choosing to waste development time on writing and maintaining tons of convoluted loops.

    So I've just released Smooth.Slinq, a faster-than-Linq-and-allocation-free, Linq / Scala inspired enumeration API for Unity.

    Asset store link

    Unity forums post

    Smooth.Slinq and Smooth.Base documentation (base docs include basic types like Option, Tuple, etc).

    Support forums
  • @Smooth_P This looks very promising. I have been looking forward to something like this for a long time.

    I would like to see how it looks in actual code though; could you provide some simple examples?

    Suppose I wanted to back my entire library with your Slinq; what would be the steps?

    Can it be hidden from users of my library that I use Slinq?
  • edited
    Generally, for one-liners the API is the same as LINQ's, but with more methods available and relaxed delegates that support variance. Because C# lacks advanced implicit conversions you simply have to Slinq() your collection.

    (Out of the box, foo.Slinq() extension methods are provided for IList<T>, LinkedList<T>, and Enumerable<T>, including descending ordering and other options where applicable. Smooth.Slinq.Slinqable defines these extensions and also has methods for creating simple numeric sequences, generic sequences generated by a delegate, empty Slinqs, repeating Slinqs, etc.)


    So something like:

    var countLinq = list.Where(x => x > 0).Count();
    var countSlinq = list.Slinq().Where(x => x > 0).Count();



    Note that one of the ways that real-world LINQ usage leads to allocations is due to closures. To avoid closures Slinq provides method signatures that take an extra, user-supplied parameter that is passed to any delegates and can be used to capture state:

    var y = 42;
    
    var countLinqClosureOverY = list.Where(x => x > y).Count();
    var countSlinqClosureOverY = list.Slinq().Where(x => x > y).Count();
    
    var countSlinqAllocationFree = list.Slinq().Where((x, p) => x > p, y).Count();
    // y is passed to the delegate as the second parameter



    A Slinq is equivalent to an IEnumerator, not an IEnumerable, so manually looped Slinq usage would look like:

    var slinq = list.Slinq();
    while (slinq.current.isSome) {
    	//
    	// Do stuff with slinq.current.value
    	//
    
    	slinq.Skip();
    	// or, slinq.Remove(), if supported by the underlying collection
    	// or, slinq.Dispose(), if you're done early and don't care about the remaining elements
    
    	//
    	// There are also other partially consuming operations like:
    	//
    	// Skip(int)
    	// SkipWhile(predicate)
    	// SkipWhile(seed, aggregationFunc)
    	//
    	// as well as corresponding Remove() counterparts.
    	//
    
    	//
    	// You can also use a fully-consuming operation on the remainder of the Slinq, like: 
    	//
    	// var maxRemaining = slinq.Max();
    	// do stuff
    	// loop will end as slinq will now be empty
    }
    
    
    // Cleaner than a where loop, with no dangling var:
    
    for (var slinq = list.Slinq(); slinq.current.isSome; ) {
    	// Do stuff
    }


    Of course, Slinq has ForEach() methods, so those are also one-liners. "Loop syntax" is only needed for times when you want to consume part of a list with a particular operation and leave the rest for other operation(s).


    Oh, I guess I should have linked the following doc for people who are unfamiliar with Option semantics (slinq.current is an Option<T>, as are the return values from potentially empty operations like First, Last, Max, etc. An Option<T> is a value type that can be thought of as a far more robust version of Nullable<T> and has two fields, public readonly bool isSome, and public readonly T value, as well as various chainable, LINQ-like methods.):

    https://www.smooth-games.com/docs/smooth-base/

    Documentation for Option<T>, Tuple<T1, ..., Tn>, etc are there.
  • @Smooth_P welcome and congrats on the launch of Slinq, it looks like it must have been quite an undertaking :)

    Your Asset Store page has a couple of screenshots of the profiler, and makes claims about performance relative to Linq. Do you perhaps have an article/blogpost/whatever that discusses theses in more detail? Your URL just goes straight to your product forums.

    And on the topic of your website and product forums, you may want to have a chat to @hermantulleken about their experiences launching Grids, and what did and didn't work from a marketing perspective. Time for a Humble Unity Plugin Bundle, methinks? ;)

  • @Smooth_P Thanks for the additional info. I will definitely have a look at it. Would you care to share some of your design philosophy behind your decisions? For instance, why not make extension methods to IEnumerable that automatically converts to Slinq (it may be a dumb question, but not immediately obvious to me).

    (The reason I ask is that I'm actually solving similar problems (but for different reasons). For instance, the AOT compiler won't generate all the code necessary to do method calls like Contains, Remove, etc. on Lists and IEnumerables that contain value types. So I too have to roll our alternatives, and I have not figured out the right "place" to solve the problem. It would be interesting to hear how you approached it.)
  • edited
    @mattbenic

    I haven't actually looked at the LINQ code, but by simple profiling, "reasonable" assumptions, and various things I've read, I figure that for constant space operations, LINQ and Slinq have a near-identical amount of overhead, with LINQ using virtual methods while Slinq uses delegate invokes. The minor performance gains likely come from the fact that Slinqs are inherently stack-based and thus do not require iterator allocations and have less dereferencing which may lead to slightly faster IL. Also, the "bottom layer" Slinq will generally know how to manually enumerate its underlying collection, which saves a method call on each enumeration step.

    But these are tiny differences. I think it's fair to say that LINQ and Slinq are equivalent speed-wise for constant space operations, with Slinq having a small advantage in simple benchmarks due to micro-optimizations. But the difference in raw speed is likely insignificant in real world scenarios, especially when compared to the cost of temporary allocations / deferred GC.

    For linear space operations, well, LINQ has to allocate a lot of space! And AFAIK, it uses List<T>s, which will need various numbers of resizes depending on the size of the data, as well as the data composition for grouping operations.

    Slinq is not currently optimized for raw speed, it uses singly linked lists for grouping and sorting in order to use a predictable amount of space that is directly linear to the size of the data and to facilitate "zero-copy" operations on pooled values. So Slinq does more method calls (and locking) to build it's storage, but once the pools are warm these calls are non-allocating.

    Slinqs are also enumerators, not enumerables, which potentially allows for various optimizations since you can only enumerate once. It's also possible that some parts of Unity's version of LINQ are simply implemented badly. I mean, would that be surprising in any way? ;)

    At any rate, I don't know exactly where all the speed gains come from for linear space operations, and some of it depends on the data, but they are there, and they are significant.
    Thanked by 1hermantulleken
  • edited
    @hermantulleken

    AOT is a bitch. ;)

    I have another asset called Smooth.Compare that I use to eliminate comparison allocations and prevent JIT exceptions. Basically it's a replacement for System.Collections.Comparer/EqualityComparer<T>.Default that lets you register comparers or automatically creates them for various types using emit if JIT is available.

    All my code that performs generic comparisons use the "Smooth.Collections" comparers which are explicitly registered for all known-to-be-compared value types when running in an AOT-only context. For operations like Contains on a List<T> you unfortunately have to supply a comparer on every method call, but you could potentially do this with an extension method.

    And the reason I don't have implicit conversions to Slinq is that a Slinq enumerator of element type T is actually a Slinq<T, C> where C is a context type. And that C can't be filled in implicitly with the features available in C#.

    Edit: Clarification on JITException avoidance.
  • @Smooth_P Thanks for the additional info (it would definitely be cool if you put some of that on your web site). You products are compelling. Do you provide source code or binaries?
  • edited
    @hermantulleken

    My stuff on the Asset Store comes as code, and I have no plans to have obfuscated DLLs or anything else. I'm a coder and understand how important having the code is. I absolutely HATE using things that come in DLLs and being unable to modify / fix things or even see how they work when the documentation doesn't give enough detail (see: Unity, uLink).

    I'm also currently working on packaging my "Foundation" classes in a free to use, free to redistribute way. These would have hooks for Smooth.Compare and Slinq for people who have them, otherwise it would fallback to LINQ and/or the default comparers (or manually defined ones, which would give users a reasonably clean way to avoid JITExceptions).

    My experience with the Asset Store so far tells me it would be at least a week before such a thing were posted even if I submitted today (and I need to do a bunch of documenting, grr!)... But if you want to check it out I can get you the code directly.

    Edit: Actually, here is the code, some of it has been recently written and/or refactored, so let me know if you notice anything strange:

    https://www.smooth-games.com/unity/foundations/smooth-foundations.unitypackage
    Thanked by 1hermantulleken
  • edited
    I've just submitted new versions of Smooth.Foundations, Slinq, and Compare to the Asset Store. Foundations can also be downloaded here and is free to use, free to redistribute. Online documentation for is available for Foundations, Slinq, and Compare.

    Changes include a Slinq API that is more drop-in compatible with LINQ, and "hooks" in Foundations that allow compiling against Slinq when available with a fallback to LINQ. Foundations also includes some useful embellishments to the LINQ API that work on various IEnumerable<T>s to help accomplish this.

    Foundations can use Compare or fallback to manually set comparers. This gives library writers a way to manually circumvent JIT exceptions, potentially using Compare to quickly and easily diagnose potential problems during debugging but distributing without it.

    I've also substantially reduced the prices of Slinq and Compare... it seems what would be considered a ludicrously cheap productivity / performance boost in the real world translates into "OMG too expensive, I'll just keep doing things the hard way and/or spiking FPS" in the zero-income-zero-users-to-please-and-zero-value-on-time world of Unity development. :D

    Anyway, check it out if you get the chance and let me know if you find it helpful!
    Thanked by 1hermantulleken
  • it seems what would be considered a ludicrously cheap productivity / performance boost in the real world translates into "OMG too expensive, I'll just keep doing things the hard way and/or spiking FPS" in the zero-income-zero-users-to-please-and-zero-value-on-time world of Unity development.
    Yeah, welcome to selling to hobbyists ;)

    I've been looking through the docs for Foundations, and I see you've re-implemented a lot of standard classes (such as Tuple) that I wouldn't have expected. Why is this? Why would one use these instead of the .Net library equivalents?
  • edited
    mattbenic said:
    I've been looking through the docs for Foundations, and I see you've re-implemented a lot of standard classes (such as Tuple) that I wouldn't have expected. Why is this? Why would one use these instead of the .Net library equivalents?
    System.Tuple<T1, ..., Tn> is .Net 4 and is also a class. Nullable<T> restricts T to a value type and can't be nested otherwise a lot of the extra functionality of Option<T> could be supplied by extension methods (with an extra pass-by-value). The delegates are there because in Unity's version of .Net, Func<> and Action<> are invariant (the .Net 4 versions are variant).

    And the Comparer stuff is there to prevent allocations and JIT exceptions. Actually Smooth.Collections.Comparer<T> and EqualityComparer<T> aren't currently in the docs because I forgot to change a #define when building the html. I've been meaning to rebuild the docs but that would imply resubmitting the assets and I've already been waiting for a while for approval and don't want to back to the end of the queue and wait another 2 weeks. 8(

    Edit: Updated the online docs and unitypackage for Foundations. The new docs also include the Slinq fallback classes. And now that I look at the docs with fresh eyes I realize there are still some things missing, which I will probably update later tonight.
  • edited
    @Smooth_P I have been wanting to have a proper look at your packages, and will very soon. It does look very cool, and I like that the free ones can be redistributed. I have also had some wacky ideas about using your LINQ version (our Grids package use LINQ alot...) but I will PM you once I had a decent look.

    About selling to hobyist and and selling time to developers, I can give you what we experienced (obviously our tools differ quite a bit, and this is just one data point, so take it with the required amount of salt).

    We had a similar experience with selling our Grids package. It seemed so obvious that it will save developers tonnes of time (and in your case, the problems your libraries solve are even more fundamental). The problem comes in with making them believe it. When you sell a bomb explosion, anyone can look at a video, see immediately it's what they want, and buy it. When it comes to abstract tools, even professionals can't make a judgement so quickly. You have to wade through code and docs, and even then it's a bit iffy whether it will actually work. And a bomb explosion is easily replaced. But a library that sits at the core of your games is a way bigger risk - what if you die? I think the trick to realise is that you are in fact not selling time - what you are really selling is trust.

    Our main selling point is also "time" on paper - but people did not really believe it. So we made 30 games in a month to show people that it really is true. It's been our biggest driver of sales since. (It also allowed us to get some coverage, make a video that is watched way more often than our official "marketing" video, do some write-ups on our experience; it gave us a tonne of extra examples which users ask for all the time, etc.)

    Maybe what you can do is do a massive (make it remarkable, not just a few graphs) benchmark project. Design a whole bunch of compelling tests that shows performance (I bundle memory and GC and all that with this) to show how well your software does compared to the built-ins. (I would take each of the LINQ operators in turn, figure out a way to show how the GC can kill you using the built-in version vs. the Smooth version and show it!) Or if you wanna sell LINQ, have a productivity competition between a the two methods ran over a time period.

    It's also surprising how much effort we spend on "marketing" compared to extending and maintaining the library. (We bunch making examples for our web site, doing support, and such all part of marketing. It's a rather pleasant form of marketing - we make lots of cool stuff. It's certainly not making ads (which we tried, and found not to work)).

    There are also other smaller things you can do to get more trust in you as professional, and your tool-house as a company. Things such as blogging about LINQ and other related issues problems; using a physical address, real name and personal email on the site; be around in other places (maybe you are already)... these can all help to make people believe you know what you are doing, see where you are coming from, etc. and give them something tangible to give them the feeling you will stick around.

    Well these are just some random thoughts.
  • edited
    Well, I'm not a marketer, and spending time on marketing and example projects would take time away from my strengths (writing APIs and code!), and isn't something I'm interested in doing.

    I wrote Slinq and Compare because they were incredibly useful for me, and the fact they they were stimulating technical problems added to the motivation! But given the sales figures it doesn't seem worthwhile to continue trying to sell them. Even if things ramped up quite a bit, I'd be much better off getting a job in the real world, solving interesting problems all day long, and working on my game as a hobby. I'd also have a lot more money to pay for art if I wanted to. :)

    Plus, I'm working on refactoring my input layer, which isn't just for bindings, but is a clean, extensible, OO system for both managing user level inputs and converting them into the format you need / want for your game, and it would be *so* much cleaner and easier if I could rely on Slinq and Compare being available. Now, this asset would have straightforward and obvious "this does X, and you know you need X" value, so I figure it wouldn't be as reliant on marketing. Getting it out there and in use could spur sales of the other assets, but the pain of writing it without them is just... well, painful...

    So I've decided to release Slinq and Compare under the MIT license. The mindshare and branding effect of giving the pure productivity and technical-problem-solving assets away for free will hopefully (!) provide more value for me than the potential sales.

    Direct download links to the assets and documentation are available on the Smooth-Games forums.
    Thanked by 1TheFuntastic
  • That's both sad to hear (that you're not making enough off it) and great to hear (that you've decided to MIT it). Good luck, I hope this brings you the exposure you need to get something else off the ground :)
  • It's not clear from that post if the author thinks this is what's currently happening (which seems incorrect, based on opening up UnityEngine.dll in ILSpy), or if it's a change happening in Unity 5.
  • edited
    (Grrr I made a mistake sorry about this).
  • edited
    Tripple post 0.0
  • This sounds like quite an exciting shift in unity compiler architecture:

    http://blogs.unity3d.com/2014/05/20/the-future-of-scripting-in-unity/
  • This sounds like quite an exciting shift in unity compiler architecture:

    http://blogs.unity3d.com/2014/05/20/the-future-of-scripting-in-unity/
    Really exciting stuff. I wonder what their timeline is to get everything shifted over to it? Other than WebGL it looks like it might only be a Unity 6 thing. I'd also be willing to bet by the time they all platforms over to this they drop plugin based web completely.

  • What? So the web player won't need a plugin, just the native browser stuff? And it'll be just as fast? Really?
Sign In or Register to comment.