Don't constexpr All the Things - David Sankel [CppNow 2021]
Вставка
- Опубліковано 11 лип 2021
- #Boost #Cpp #CppNow
Slides: cppnow.org/history/2021/talks/
CppNow Website: cppnow.org
CppNow Twitter: @CppNow
Streamed & Edited By Digital Medium Ltd: events.digital-medium.co.uk
------
The constexpr metaprogramming model, which involves annotating code with indicators of compile-time or run-time suitability, has been steadily increasing in capabilities since its debut in C++11. These efforts have been in support of the stated goal of making metaprogramming accessible to even novice developers. Unfortunately, the result is a complicated and unintuitive programming model. Elsewhere, an extremely compelling alternative that is intuitive and more powerful has been implemented in Sean Baxter's circle compiler. This talk describes the limitations of the constexpr metaprogramming model and introduces its persuasive alternative, circle metaprogramming.
------
David Sankel
David Sankel is a Software Engineering Manager/TL at Bloomberg and an active member of the C++ Standardization Committee. His experience spans microservice architectures, CAD/CAM, computer graphics, visual programming languages, web applications, computer vision, and cryptography. He is a frequent speaker at C++ conferences and specializes in large-scale software engineering and advanced C++ topics. David’s interests include dependently typed languages, semantic domains, EDSLs, and functional reactive programming. He is the project editor of the C++ Reflection TS, a member of the Boost steering committee, and an author of serveral C++ proposals including pattern matching and language variants.
------
May 1, 2022 - May 6, 2022 - Aspen, Colorado
-------------------------
---
*--*
--- - Наука та технологія
![The Worst Best Practices - Jason Turner - [CppNow 2021]](/img/n.gif)
Seeing all of this cool stuff and realizing that we won't probably have it in C++ made me sad :(
Now we just need to get Sean to open source it already. :)
This is a very well laid-out and presented talk. I only wish it would be about C++ that we can use today in our projects. I want to say: don't constexpr all the things, consteval all the things. constexpr really is too confusing, especially if all you want is to actually write compile-time-only code.
The ability to express code that _can_ be runtime or compile time is extremely important. I do wish that consteval was added in a much earlier version of the standard, though, because the inability to reason about metaprogramming in C++ with confidence for so many years has tremendously damaged the reputation of this language.
@@Cons-Cat Old comment, but I feel the need to disagree anyway. If you reach for compile time computation as a tool, I believe you've already made the decision that this is something that must happen at compile time. It's part of the solution to a problem. So if the tool to achieve that solution doesn't guarantee anything, it is not the right tool for the job. From that perspective, constexpr was starting in the wrong end. consteval should have come first. FWIW, in my humble opinion.
I really enjoyed this presentation and support the message.
Glad you enjoyed it!
This looks really great!
PHP!!!
Great talk, thanks for this
Glad you enjoyed it!
I remember reading about circle what feels like years ago. It struck me as being similar to D as far as the meta features which are far superior to C++. Also if I remember correctly the name circle came from thinking about compile/run time, that, the normal features of programming run along the X axis but Circle could be "rotated" so that programming could be "done along the y axis" (ie compile time). I think I am remembering this right.
Exactly.
no such thing as xexpressionx or constexpr etc, ts just machine, no nerdx or expx about it, cepitxuxyuax, say, can say,expx etc infix any nmw and any s perfect
Arbitrary code execution at compiletime is great, being able to have an interactive element like a commandline or even better yet a GUI would be heavenly for some applications.
The codebase i am working on is modelling physical layouts with a lot of interconnections - and they all have to be modelled in some way. With compile-time GUI we could visualise those connections, interprocess-links and interdependencies and check or even modify them. every other week i get asked by somebody why his new processplugin does not connect correctly to the subsystem or why the messages he is simulating are not arriving at the correct handler and most of the time it is some config-error that is hard to spot as you have to manually look through several config-files. (The config-layout is in general easy to use and makes sense, but it can be a lot of work to check if everything is setup correctly)
Amazing talk, the dramatizations are priceless. I must say though that cross-compiling is not just a "10% use-case"! It's the entire video games industry. Pretty much every games studio is compiling on Windows, natively with MSVC when targeting PC, cross-compiling with MSVC when targeting Xbox, and cross-compiling with Sony's own fork of clang when targeting PS5. Quite a problem if you support neither Windows nor cross-compiling.
I saw this idea of just executing code at compile-time first with the jai language, and it was a total aha-moment. Of course you can just execute any arbitrary c++ code at compile-time, that's literally what the compiler itself does. Once this idea clicked, it just seems like the obviously correct thing to do.
The only thing I'm unclear about is how you manipulate types with circle (or this approach in general). So, the more classical tmp approach conditionally instantiates different types. Since a normal function can't return a type, I don't think this would work too well with circle, and force some mix of tmp and circle. But probably they thought about this and have some solution.
My biggest worry is just the fact that the compiler is a standalone project. C++-compilers are notoriously hard to maintain, and I just don't see how this niche project won't just fall behind massively in the coming years. For serious use, I think depending on an existing compiler to do the compilation for the runtime code is absolutely required if this wants to see some real use.
There are tons of ways to manipulate types, but the main one relevant to this video is to use meta control flow inside class definitions.
template
struct tuple {
@meta for(int i : sizeof...(Ts))
Ts...[i] @(i);
};
This for loop is executed at compile time (during class template instantiation). It walks over each member of the parameter pack and declares a new data member called @(i) with type Ts...[i]. @(x) is an operator that converts strings or integers to identifiers. So, @(0) gives identifier _0, @(1) gives identifier _1, @("foo") gives identifier foo, etc. Ts...[i] is just the i'th element of the pack.
The data member declaration is non-meta, so it sinks through the meta scopes (like the for loop's body scope) and is attached to the inner-most enclosing non-meta scope, which is the class-specifier.
You can do meta control flow inside namespaces, function definitions, class-specifiers and enum-specifiers. The general strategy is to use C++ at compile time to get data that describes how you want to build your types, and use meta control flow to programmatically define them.
Also, this project is definitely not falling behind. The compiler keeps getting more robust, faster and more feature-rich. It compiles Boost Hana (the most challenging template library) 1.67x as fast as clang and 2.42x as fast as gcc. Follow me on twitter! I'm really open to ideas. If somebody has a good feature suggestion I go ahead and implement it.
twitter.com/seanbax/status/1411331612260945920
I think the issue is even bigger than just this. The C++ Standard Committee writes the "Guidelines" of the language specifics. C++ has many different available features and tools as it is a both a low and high level language covering multiple paradigms. It is a strongly typed language, it can be used in a procedural way, a functional way, in an OOP way, and with templates Metaprogramming way. The idea behind C++ originally is that it was C with Classes as well as having a few other languages features that C didn't have built in such as a bool type, passing & returning by reference and a few others. C++ has also evolved since then... Yet the major consensus and agreement with the C++ committee that write the standards have agreed on a single principle that has not changed since its inception. The language is to be fast and efficient, based on types, portable across multiple machines and these "rules" well more of a guideline are set to describe the overall "virtual machine" that the C++ compiler ought to be and it's left up to the compiler designers to implement their features in their own manner as long as they yield the desired results based on a specific criteria within a generalized context. This is why Clang versus GCC versus MSV versus Intel versus MIPs versus Ming, etc... possess some features of the languages while others do not and some implement it in one way where others in a different manner. The core idea behind the language is that even though there is a generalized standard, C++ is not Restrictive and doesn't force anyone to do anything within their implementations. It will allow you to shoot yourself in the foot if you don't know what you are doing and this is done by both intent and design. I can write an entire program in C++ without ever using Templates, without ever using Macros, without even ever using Dynamic Memory, nor even classes. I can use #defines just as in C for constants and functions with overloads and still make a workable program. I can also write an entire program strictly in templates without any runtime code nor the use of dynamic memory. I can write a program using Raw Pointers and Pointer Arithmetic managing my own memory or use some library's smart pointers. I can write a program in C++ that is done purely with Macros and Preprocessor Directives. I can write a program that is fully contained with an asm{} block. I can write code that is completely Object Oriented. With C++17+ I can write a program that is done using only Lambdas. I can also write a program that uses only static class member functions. I can write a program that uses any or all of the features. C++ is very powerful language if one knows not only how to use the language but also knows how to design their data sets and their algorithms and knows what type of idiom to use within the right context. I'm also 100% self taught with the language and I've used it along with various libraries and APIs to build a working 3D Graphics / Game Engine with 3D Audio. I understand most of the languages features and knowing when, where and how to use them properly is key. This isn't going to be learnt overnight especially by someone who is new to the language. This takes years of practice, experience, patience, trials and errors along with problem solving skills. You just don't have to understand the language features, but you also need to know your compiler, linker, debugger and their error reporting systems as well as the Hardware & OS that you are targeting. How about when you start to write code that is going to executed on an Intel Architecture where its byte order is in Little Endian and you are working with Unions, Bitfields, binaries while doing some bit twiddling and you need to convert it to be "Network Compatible" where the standard convention is in "Big Endian". Now when you start to work with code that is in terms of Endianness based on a specifically targeted machine, you have to know that it isn't going to be very portable or generic code. Yet with the power of templates and metaprogramming along with preprocessor directives, you can have this test to see what Architect it's going to be executed on and in which Operating System will be handling the assemblers system calls, then you can branch your code to work on the appropriate different architects and platforms accordingly. Does this mean your entire program has to be branched? No, not necessarily it just might be that one specific function where you want a variant for each to run as optimally as you can based on the machine that this function is being called on. So even when you are working in C++ you just don't need to know the language. In most cases many may not need to know Assembly but in various cases you might and this also involves you to also know the hardware and its capabilities as well as the operating system. This also implies that you have to understand what the compiler is doing under the hood as well as what your assembler is doing too and this also implies that you would want a decent understanding of the ISA that you are going to targeting. Now with modern C++ and many of the "std::" library features that are out there they have made it a lot easier to allow your code to easily be more portable and generic over the years but this is only if you choose to use the std:: library. I can write an entire C++ program without std:: and use the simple C runtime library by using fprintf, snprintf_s, malloc, calloc, etc... You as the programmer have the choice to design your program as you see fit as long as it meets the specifications and criteria to your clients. So the core takeaway is that C++ will allow you to drive off a cliff if you are not paying attention to which direction you are driving...
Very good talk! Sure the best way to give talks online.
I like all the ideas of Circle. Too bad it has no open source compiler and therefore no community or company to back it up. I am open for surprise, but I guess this goes nowhere.
And as the C++ community will not accept these kind of innovations, we have to bet on Zig, D or some other new language to become successful - or work on our own language that probably will go nowhere as well.
Keep up the great work on your talks!
Oh my god this talk makes SO much sense! Thank you! I WANT THIS :) Pls publish the compiler out :) I hope it gets into the C++ standard. Want a petition on this to be included into C++ :)
That compile time print reminds of the jai language. This is such a well presented talk, blew my MIND!
Why isn't the compiler open-sourced?
The file injection sold me
16:10 LOL :-D
36:00 PHP :)
Re security of executing arbitrary code at compile time:
How does Perl 5 or Lisp handle it? In Perl 5 "compile time" is done by the user who's about to run it, so there's really not much difference to his security whether something is done in the compiler's context or the eventual run-time context. Lisp reader macros, though, are executed at an ahead-of-time compilation so have the same issues as C++ would.
The idea of allowing "arbitrary code" refers to syntax and allowed language constructs. The issues regarding security have to do with allowed library functions and sandboxing.
You make an important distinction that gets lost on people. The ability to interpret non-constexpr functions is a language decision; general compiler security is orthogonal to that.
What a brilliantly done presentation. Well done. And Circle sounds like a phenomenal piece of work. I do worry about basing a new language on C++ though. C++ is already a ridiculously complex language that nobody understands in it's entirety and shows no sign of cleaning itself up.
ahaha this intro to dynamic allocations :D ROFL
but does it even godbolt?
I have a feeling that it will soon...
Circle is cool but i can't help feeling like i fell for a bait+switch.
PHP!
As much as I agree with the idea behind circle, we refused to include std::embed in the standard because of security and being able to know when to recompile from the build tools. You can also throw reproducible builds and incremental compilation. I mean, it can be an acceptable tradeoff for some, but I like my incremental compilations.
about std::embed:
meta-build-systems like CMake already scan the files themselves for what they include to build the dependency tree; and build-systems just check if a file changed, it shouldn't matter if it's a .cpp or a .png file
and security, well, if you don't review the code you compile, it doesn't matter if the program gets the content of your /etc/shadow at compile time by the compiler or reads it in itself at runtime
The C++ standard still refuses to acknowledge that files exist and wants to somehow tackle security in lieu of the compiler and build system vendors, that'd be hilarious if it wasn't sad. Not to mention most of the reasons to block #embed weren't even rational: "just parse better" or whatever unfounded and dismissive nonsense the poor guy pushing the idea was getting as answers. Thankfully the C comittee were more open-minded and practical and accepted embed into C23, which likely means we'll get it by default in most C++ compilers.
I't's like the C++ committee is missing every appointment with History it can, at this point.
Same with auto, there are some use-cases not to use auto.
I think this would be a security issue. Instead of the OS running your executable, it runs the compiler which in turn runs your code. The extra step does seem to be another security weakness.
PHP.
I don't agree with the goal, "compile-time programming should be exactly the same as runtime programming". They require different ways of thinking so you should be able implement differently. Also you should be able to distinguish between them.
What's really ideal is to make all code const and/or constexpr by default. But that would make C++ no longer backwards compatible.
If you do that (constexpr by default) you will not be able to distinguish what's compile-time and what's run-time, which was your first complaint
I've watched a few minutes and I can't really agree with this. I mean sure, you don't need to constexpr all the things, you just do it when it is useful to the software you're working on. I've seen the Jason Turner's talk, and I think you didn't understand the message they were trying to give: they just made an extreme example that you can move everything at compile time if you want to, but it doesn't mean that you should. But mostly, it's a fact that C++ is by design a language that gives the user a lot of power, through metaprogramming, memory management, and now constexpr. So if you can't use these tools, why are you using C++ in the first place?
Templates and metaprogramming ARE normal C++. Instead of saying they are a different language, what you really should say is: it's a part of the language I am too lazy to learn. It's really not a rocket science, and any software engineer, who passed an lc interview, should have no problem learning it. CPPCON is not a measure of complexity, there are a lot of talks there every year about more "simple" parts of the language. Neither long sentences from the standard are. Read the rules for special member functions generation. Voting down.
"Templates and metaprogramming ARE normal C++"
They are not. It is a different syntax with different rules and different behaviour.
@@ABaumstumpf It's like saying that integrals are not normal calculus because they have a different syntax than derivatives. Many programmers are really strange "professionals." If you are a mathematician who knows calculus, you can't say, "I know calculus, but not integrals; they are too complicated and have weird syntax." In programming, it's totally acceptable, for some reason, one can learn 30% of the language and put C++ on their resume. Not only that, they also can aggressively dumb down their colleagues, demanding all around to use the only subset of the programming language they managed to learn.
"We don't need a language feature"? Wut? Circle != C++. A different language is a language feature to the max. Sure, the syntax is mostly the same. But it's not exactly the same. You need specific compilers and coding style. Maybe you can read it more easily, but it's just a more of the same.
Might as well switch to Rust.
The syntax is a superset of ISO C++. It implements most of the C++ 20 standard. It uses the Itanium C++ ABI. It's as much a C++ compiler as gcc, clang or msvc. All compilers have many extensions, (which are often needed to implement the standard library), Circle just has many more than the others. Switching to Circle is not at all like switching to Rust, because you don't need to learn a new language, and you don't have to recode anything. Start with your existing project and begin using reflection and the other features.
Circle is C++ .. is just a superset of 17. They even wrote a formal proposal to consider its features for the standard. Comparing that to Rust is completely ridiculous.
Yep, exactly. It's so lol: "oh, it's so bad that metaprogramming is a second language inside C++, let's come up with a third "language" then". It's a fantastic logic.
@@TheOnlyAndreySotnikov it's not a "third langauge." It provides a context for executing normal code at compile time. You can eg parse a json with json.hpp and not have to modify anything in that library. The point is to get more use out of existing software.
@@seanbaxter1050 hi sean, I think you're doing great work. What I'm trying to say is, as long as this is not part of the C++ language and implemented in the major compilers, it's "alien". I know the rejection was a bummer, so sorry for that. But even though this looks a lot like C++, it's not standard C++.