Is syntax highlighting planned?

The integrated help in the Python script editor is a great consistent 'Prime' UX touch. Definitely one of those features that will be appreciated by those taking their first tentative steps with Python. I'm sure it will be appreciated by educators.

Hello: when the beta version 2.1.14541 (2021 04 13) is converted to the new firmware, will there be an update to CAS 1.70?
Thanks, Roberto Mioni

(04-15-2021 01:03 PM)jonmoore Wrote: With KhiCAS on the TI Nspire CX II offering the complete XCAS of desktop flavours of XCAS, I'm hoping Bernard will be free to port KhiCAS to the Prime now that it features a MicroPython interpreter.

It's mainly a licensing issue for some components that I did not write myself. KhiCAS on the Nspire is a GPL application on top of a closed-source system which has been partially root-ed with ndless. There is no such thing on the Prime, you can not compile native C third-party application and run them there ... unless someone publishes a port of ndless to the Prime of course!

(04-16-2021 11:01 AM)parisse Wrote:

(04-15-2021 01:03 PM)jonmoore Wrote: With KhiCAS on the TI Nspire CX II offering the complete XCAS of desktop flavours of XCAS, I'm hoping Bernard will be free to port KhiCAS to the Prime now that it features a MicroPython interpreter.

... unless someone publishes a port of ndless to the Prime of course!

Now you're talking!

(04-16-2021 03:37 PM)jonmoore Wrote:

(04-16-2021 11:01 AM)parisse Wrote: ... unless someone publishes a port of ndless to the Prime of course!

Now you're talking!

Linux and even Windows 10 IoT has been ported to the HP Prime G2, but it requires opening up the calculator to put it into NXP's serial download mode by shorting a pin. I suppose one could try and develop a jailbreak for those not willing to get their hands dirty, but I doubt HP would approve of this.

On the subject of KhiCAS. It brings to mind a few niggling concerns with the Python implementation on the Prime (that's been exposed to us so far). Something I had hoped for, was hooks into the native CAS system. Having a programming tool on a calculator who's key selling point is the elegance and simplicity of it's CAS without an API to program said CAS is a tad worrying. I'm fully aware that HP PPL enables the user to program the CAS but the whole point of Python is it's accessibility, relative simplicity and universal familiarity in STEM environments at all levels of education.
Having explored Python on the NSpire since it was first introduced, one of the key areas it provides a superlative experience is that that desktop software is effectively an IDE when working with Python. Using the Prime desktop emulator as an IDE doesn't compare.
Thinking about this IDE issue I was wondering if any consideration had been given to using Thonny as an IDE with the Prime. Thonny is very well know amongst educators as it hooks into Raspberry Pi/BBC Micro boards (and suchlike) and provides a simple MicroPython script editor/REPL for any boards connected to the host computer. I know that Primes can only be 'seen' on a host computer via the Connectivity Kit. But maybe there's a way to connect to Thonny in tandem. Nobodies going to want to write programs of any length on the calculator and whilst the emulator means touch typists can work at speed, it's still far from be being an ideal situation.

(04-16-2021 05:08 PM)Jean-Baptiste Boric Wrote:

(04-16-2021 03:37 PM)jonmoore Wrote: Now you're talking!

Linux and even Windows 10 IoT has been ported to the HP Prime G2, but it requires opening up the calculator to put it into NXP's serial download mode by shorting a pin. I suppose one could try and develop a jailbreak for those not willing to get their hands dirty, but I doubt HP would approve of this.

Interesting.
However, the beauty of ndless is that it's relatively unobtrusive with regards to the NSpire OS. I was only joking ref ndless. The chances of it ever getting ported is very remote. The developers may pride themselves on jailbreaking the TI hardware but they're TI loyalists all the same!

04-16-2021, 06:49 PM (This post was last modified: 04-16-2021 06:50 PM by parisse.)

I really believe that the assumption that if third parties have access to native programmation, they will necessarily begin by trying to break the exam mode is simply *wrong*. Hackspire has since several years published information about the LEDs of the Nspire CX, but AFAIK KhiCAS is the first ndless program that implements it's own exam mode and it does not break the exam mode rules.
Independent developers on a calculator do obviously not want to see the target calculator banned on tests. It is their best interest not to touch to exam mode unless it's really necessary (which was the case for KhiCAS on the Nspire CX).
My personal opinion is that it would be a win, not a loss, for HP if the Prime would provide a way to run native code.

04-16-2021, 11:35 PM (This post was last modified: 04-16-2021 11:36 PM by Jean-Baptiste Boric.)

(04-16-2021 06:49 PM)parisse Wrote: I really believe that the assumption that if third parties have access to native programmation, they will necessarily begin by trying to break the exam mode is simply *wrong*. Hackspire has since several years published information about the LEDs of the Nspire CX, but AFAIK KhiCAS is the first ndless program that implements it's own exam mode and it does not break the exam mode rules.
Independent developers on a calculator do obviously not want to see the target calculator banned on tests. It is their best interest not to touch to exam mode unless it's really necessary (which was the case for KhiCAS on the Nspire CX).
My personal opinion is that it would be a win, not a loss, for HP if the Prime would provide a way to run native code.

Well, it's not me who needs to be convinced.
To be fair, native code support does require design/development/QA time and manpower to implement, which until recently seemed to be in rather short supply for the HP Prime. It does carries non-zero amounts of risk that can be mitigated against at an extra cost. I doubt HP would be willing to go down that road. Instead, maybe we would ask for something more realistic.
Personally, I'd vote for WebAssembly support. It's an open industry standard, it's sandboxed by design, several open-source implementations are available under different licenses (Wasm3 is under the MIT license for example) and the LLVM toolchain supports it as a backend. People who keep complaining about KhiCAS on the HP Prime would finally be able to run it (even though it's mostly just giac which is already available by pressing the CAS key, along with some extra bits and a bespoke UI).
It's not quite as fast as native code execution especially if it's just interpreted, but most program and libraries can be ported to it as-is and the HP Prime has ample performance headroom to handle that. We don't even technically need HP's support since a WebAssembly interpreter could be written in HP PPL, although performance would be rather poor when compared to an official solution.

(04-16-2021 11:35 PM)Jean-Baptiste Boric Wrote: Well, it's not me who needs to be convinced.

I do not try to convince you, I present arguments to all readers, including the HP team, that the risk that someone would break exam mode if native code is available is over-estimated. Unless I'm mistaken, if you look at the history of exam mode security attacks on calculators, they come from exploits of OS bugs, and not from native code execution.

Quote:To be fair, native code support does require design/development/QA time and manpower to implement, which until recently seemed to be in rather short supply for the HP Prime. It does carries non-zero amounts of risk that can be mitigated against at an extra cost. I doubt HP would be willing to go down that road. Instead, maybe we would ask for something more realistic.

Casio has support for native code (addin) on their calculators. I don't think it's unrealistic to hope the same on HP. The HP Prime has a much more capable hardware (no 2M upper limit for example), that could attract more developers and make the Prime more attractive for all users.

Quote:Personally, I'd vote for WebAssembly support. It's an open industry standard, it's sandboxed by design, several open-source implementations are available under different licenses (Wasm3 is under the MIT license for example) and the LLVM toolchain supports it as a backend. People who keep complaining about KhiCAS on the HP Prime would finally be able to run it (even though it's mostly just giac which is already available by pressing the CAS key, along with some extra bits and a bespoke UI).

It's more than just some extra bits. My main concern is that I can not publish myself a fixed firmware when someone detects a bug in the CAS, or if I want to bring improvements. For example, I'm currently improving symbolic integration code in giac, I will probably publish a revised nspire version in a couple of weeks but I don't know when it will be available on the Prime.

Quote:It's not quite as fast as native code execution especially if it's just interpreted, but most program and libraries can be ported to it as-is and the HP Prime has ample performance headroom to handle that. We don't even technically need HP's support since a WebAssembly interpreter could be written in HP PPL, although performance would be rather poor when compared to an official solution.

wasm is already supported by giac with emscripten (e.g. in Geogebra, Xcas for Firefox, Tableaunoirxcas). But it's not as simple as you say to port libraries. It's easier to port libraries with a cross ARM toolchain. And of course you will have a performance penalty. With a good wasm implementation, mean performance loss is a factor of 3 for giac (e.g. inside Firefox), but it can be much more for bigint computations (7 or 10 times for Groebner basis computations for example). Wasm3 would probably be at least 10 times slower (mean) and up to 50 times slower for large computations and that's significant for CAS.
And it's not clear that adding wasm support would require less work than adding native code support...

(04-17-2021 06:36 AM)parisse Wrote: Casio has support for native code (addin) on their calculators. I don't think it's unrealistic to hope the same on HP. The HP Prime has a much more capable hardware (no 2M upper limit for example), that could attract more developers and make the Prime more attractive for all users.

Casio is willing to support native code, which is a good thing. HP hasn't shown any sign that they would be willing to reconsider their position on the matter ever since the original HP Prime was introduced back in 2013. Given that constraint, I'm searching for a solution that has a non-zero probability of happening.

(04-17-2021 06:36 AM)parisse Wrote: It's more than just some extra bits. My main concern is that I can not publish myself a fixed firmware when someone detects a bug in the CAS, or if I want to bring improvements. For example, I'm currently improving symbolic integration code in giac, I will probably publish a revised nspire version in a couple of weeks but I don't know when it will be available on the Prime.

Although this was somewhat in jest, I failed to consider that this would be a solution for providing updated giac builds faster than the official channel. My bad.

(04-17-2021 06:36 AM)parisse Wrote: wasm is already supported by giac with emscripten (e.g. in Geogebra, Xcas for Firefox, Tableaunoirxcas). But it's not as simple as you say to port libraries. It's easier to port libraries with a cross ARM toolchain. And of course you will have a performance penalty. With a good wasm implementation, mean performance loss is a factor of 3 for giac (e.g. inside Firefox), but it can be much more for bigint computations (7 or 10 times for Groebner basis computations for example). Wasm3 would probably be at least 10 times slower (mean) and up to 50 times slower for large computations and that's significant for CAS.
And it's not clear that adding wasm support would require less work than adding native code support...

To summarize my points on why I think WebAssembly is a good candidate here:

• Integrating an off-the-shelf WebAssembly runtime library should be as hard as integrating the MicroPython runtime. WebAssembly doesn't require a shell, a script editor or complex UI.
• Native code is not portable between the different platforms supported by the HP Prime ecosystem (physical calculator, smartphone/tablets apps, desktop simulators...). Separate add-ins builds are required for every supported platform. Every supported platform requires platform-specific glue. Side-loading native code is not allowed on iOS by policy. For an ecosystem that offers a consistent feature set across all platforms, native code would feel like an afterthought at best.
• WebAssembly is sandboxed by design, which is important not only for security but also for stability. Native code requires complex technical solutions for that (at the minimum privilege separation with a separate address space).
• It's an open industry standard supported by the LLVM toolchain. Most program and libraries can be ported to it, which would open up the HP Prime ecosystem by enabling ports of existing software, including KhiCAS.
  

Indeed, it's not as fast as native code, especially if an interpreter-only runtime is integrated. However, I believe that for most usages it would be fast enough on the physical calculator. If people really want better performance in order to run Quake on their calculators at an acceptable framerate, HP could change the WebAssembly runtime for one with JIT or AOT compilation support in a later release without breaking compatibility. It's not a technological dead-end.
Yes, ideally HP would support native code and we wouldn't even be having this discussion in the first place. If you can convince them to change their mind on that topic then good, in the meantime I'm advocating for a technical solution that gets us most of the benefits for native code support without actually having native code support, so that it has a non-zero probability of happening.

04-17-2021, 01:09 PM (This post was last modified: 04-17-2021 01:16 PM by jonmoore.)

My own view on HP's position regarding the matter of native code 'addins' is that it should be given serious reconsideration.
Whilst I'm not an educator I provide consultation services to the education market here in the UK and my partner and I have four teenagers at various school stages (pre-GCSE to A-level). On that basis, I own all Casio calculators (fx-991EX, fx-CG50 & CP400, the last of which is a subscription to the ClassPad service as the hardware is painfully slow), the Nspire CX CAS II, and a Prime (as well various HP/SwissMicros collection hardware, of which, the 50g is still my most used of the bunch). That's my personal booty, the kids have their own hardware (only one of them has the Prime hardware, but all have the iOS versions of both the Prime and ClassPad apps, thanks to Apples family sharing feature).
The reason for Casio's success in European markets (especially the UK) has nothing to do with 'cartels' (which is often the accusation thrown at TI in the US), it's simply down to their strategy of building hardware solutions specifically to education sector requirements. The 991EX stands out in particular in terms of its feature-set, as it's all that's required all the way to A-Level standard. Not bad for a calculator costing £20. Sure, its diminutive hardware can only solve toy textbook problems but it fits the UK curriculum perfectly.
As to the CG50, even though its graphing features are less powerful than either the Prime or Nspire, its UX surpasses both. And it's a superior calculator for A-Level Statistics in terms of its UX. The fact that Barnard has a near fully functional version of XCAS on the platform lift's the usefulness of the CG50 platform to greater heights (such a pity it's limited to a 2Mb app size limit). Casio sells the CG50 directly to UK students through examination boards for approx. £75 and that of course adds to it's success.
As an outside observer, HP's paranoia with regards to 'Exam Mode' appears to have been created by the fact that they mistakingly launched the Prime without an Exam Mode, and it's CAS features restricted its use from most exam board requirements in the UK (that I know with certainty) but I believe that situation was much the same with many European exam boards other than the International Baccalaureate. The lack of an Exam Mode was fixed in later OS updates but unfortunately, the damage was already done in terms of the Primes reputation amongst educators. I don't see the Prime's fortunes changing in the UK until/if a next-generation model is released (that gets the OS correct from the getgo). The years between the G1 and G2 models gave TI a chance to update their hardware (unfortunately the OS is still slow, so the processor upgrade provides smaller performance increases than otherwise expected). Yet again KniCAS to the rescue, as it's far better able to leverage the hardware.
Python provides the HP team with a new opportunity to score points with educators but the implementation, as we see it right now, provides little advance over Python implementations on other calculators. It's better than the Python implementation on Casio's CG50 but Casio have far bigger plans for Classpad as a platform in the coming years, so it will be interesting to see if they even bother launching a successor to the CP400 (the CP500 is still using the 400's painfully slow 2013 hardware, and only differs by the fact of not featuring the 400's 'softkey' QWERTY keyboard).
The Prime continues to be the best and most immediate CAS calculator on the market but that's not enough to make it succeed with educators. Where HP succeeds is with end-users. The Prime has a great reputation amongst A-level and university students doing courses where exam board derived restrictions don't apply. This reputation can only be furthered by the availability of KhiCAS and suchlike, as native code solutions. In lieu of that, I hope HP have further plans for their Python implementation than we've yet been made aware of. It would be a pity if Python on the Prime was perceived as nothing more than a 'keeping up with the Jones's effort'. The Prime hardware is far too good for its Python implementation to not leverage said hardware more fully.

I don't think native code has a near 0 probability of happening, otherwise I wouldn't discuss.
I'm not against a wasm interpreter for those who are happy with a wasm solution, but I would not be happy for speed reasons at least and I'm probably not alone. I would certainly not spend time myself to port KhiCAS on the Prime with a slow wasm solution. On the other hand, with native code support, I would certainly port KhiCAS to the Prime, this would be the most powerful calculator platform where I could do that. And this could serve as a wide-testing area for giac (optimizations testing, improvements, new features, including integration of other projects like multiprecision floats and QuickJS), before those with compatible license could be integrated in Prime firmwares.
The main reason not to have native code is the fear that Prime could be banned because of exam mode attacks. Like every decision, there are advantages (e.g. for the CAS) and risks to bring native code support. Zero risk does not exist but I believe it is far over-estimated. Look at the history records for other calculators:
* Casio has native code support since about 10 years: AFAIK, there hasn't been any exam mode concern with addin
* ndless brings native code support on the TI Nspire since 2011 according to wikipedia: there hasn't been any exam mode concern
* there is asm support on the TI83/84. There were exam mode concerns on the TI83, but related to OS bugs, not to asm code.
Moreover the probability to see someone play with exam mode is proportionnal to the user basis size in schools and the Prime has certainly less users than more affordable models like B&W Casio fx 9860 series with native code support.

04-17-2021, 04:17 PM (This post was last modified: 04-17-2021 04:39 PM by Jean-Baptiste Boric.)

(04-17-2021 01:25 PM)parisse Wrote: I'm not against a wasm interpreter for those who are happy with a wasm solution, but I would not be happy for speed reasons at least and I'm probably not alone. I would certainly not spend time myself to port KhiCAS on the Prime with a slow wasm solution.

I've made some quick benchmarks with eigenmath on my workstation (I couldn't be bothered to make sense of giac's build system to spit out a wasi-compliant wasm build). For reference, wasmtime is an Apache 2.0-licensed JIT runtime and wasm3 is a MIT-licensed pure interpreter. Both could be embedded in the HP Prime calculator, although wasmtime might not be available for the HP Prime G1 CPU and also for platforms that forbid JIT compilation, like Apple's iOS.

What do you think of these performance numbers?

thanks very much Tim & Cyrille.
Hoping for a MacOS build at some point…
(or is it OK to merely update my physical Prime, and use the older CK?)

04-17-2021, 06:20 PM (This post was last modified: 04-17-2021 06:22 PM by parisse.)

I'm afraid you can not deduce much from the eigenmath timings.
For example, 100000! seems really slow with eigenmath, with giac it takes 0.01s natively and 0.35s in wasm on my 5 years old Mac
session Xcas
For is_pseudoprime(2^4253-1), I get 1s natively and 41s in wasm.
The large ratio is probably a consequence of GMP native optimizations.
For this example, I'm surprised by the wasmtime you get with eigenmath compared to giac wasm time inside Firefox: if there is no bug in eigenmath for wasm, it would mean that multiplication and remainder of big integers are faster in eigenmath than with GMP with generic code, but that's not what we observe for 100000! It would require more benchmarking with arguments of various bitsizes.
Anyway, large integer operations in giac with GMP are at least one order of magnitude slower in wasm wrt native on x64. I don't know if this is still true for ARM, but I guess that GMP has also ARM native optimizations.
In my experience floating point operations performance is acceptable in wasm inside Firefox (a speed ratio of about 2 or 3).
BTW, you probably do not need to compile giac yourself, giac.wasm is available from here and you can easily benchmark with time() in simple.html. If you want to compile yourself, you can download emgiac (emscripten 1) or emgiac (emscripten 2)

04-17-2021, 07:47 PM (This post was last modified: 04-17-2021 07:57 PM by Jean-Baptiste Boric.)

I'm aware that these numbers won't translate directly to giac, but they do give an idea of the overhead of WebAssembly vs. native for general-purpose code (eigenmath doesn't use any libraries, it's self-contained). I expect optimized libraries like GMP would take a greater performance hit, but maybe native versions from the firmware could be exported for consumption by WebAssembly modules if required for performance reasons.
For isprime(2^4253-1), according to eigenmath's source code it looks like Miller–Rabin primality test (don't quote me on that, I'm loosely correlating things with the help of Google). Maybe it has been tuned differently? Maybe GMP's generic routines don't mix well with WebAssembly inside the toolchain? Maybe Firefox's JIT engine has a hard time dealing with it? Maybe it's a combination of the above or some other factors? Who knows, but I'm way out of my comfort zone here to make any conclusions.
Anyways, I'm probably not the right person to benchmark computer algebra systems in the first place since I suck at algebra... giac should be benchmarked too, preferably with both a pure WebAssembly build of GMP and an exported native build of GMP if it's doable, to have an idea of the performance levels we can expect with a WebAssembly solution. However, it sounds like GMP could be a sizable performance bottleneck, but unlike a web browser context here we do have more options to address that.
Note that we haven't talked about the implications for exam mode, but that is a whole another set of problems I'm not looking forward to discuss.
I can't reuse your wasm builds of giac for these tests because these runtimes target WASI (WebAssembly System Interface) instead of a web browser. The imports simply don't match up, a specific port (or at the very least build) is required.

A good test for eigenmath isprime would be to look at different prime bitsizes n and see if it behaves in O(n^3) (expected runtime for Miller-Rabin). This can easily be done by generating primes with nextprime in giac. For example in giac
n:=1023; p:=nextprime(2^n):; time(is_pseudoprime(p)); [0.011s]
n:=2047; p:=nextprime(2^n):; time(is_pseudoprime(p)); [0.076s]
Multiplying the bitsize by 2 should require about 8* more time to run the Miller-Rabin test, this is what we observe here with giac.
BTW, if you could run the same tests as above with giac on your computer, this would give us an additional information, the speed ratio betwen your computer and mine. Asssuming you have a more recent computer than mine, I bet that giac native is more than 10* faster than eigenmath native on x64 for your 2 integer benchmarks. This would confirm that a wasm version of giac would be at least 10* slower on big integer problems (and perhaps up to 40* slower) than a native version.
As for exporting a native compiled GMP version, this is certainly technically not easy. Moreover GMP can not be integrated into the HP Prime firmware because it is licensed under the LGPL. Well more precisely it could be, but then HP would have to bring a way to build new firmwares for people who want to modify GMP, and that's much more than supporting native code.
If you can not use the same wasm output for your benchmarks than what I generate with emscripten for the browser, then well it means wasm is not really that universal... Compiling GMP to wasm is not easy, recompiling it with another toolchain would require work, and that work must also be done for MPFR and MPFI (at least). Compiling GMP/MPFR/MPFI to ARM native is much easier and is already done for the Numworks and Nspire.
As you can see, there are many arguments in favor of native code support.
This discussion was interesting, I will probably explore a little more if GMP is the right solution for big integer support for wasm in the browser.

Bernard, this question isn't related to the Prime, but instead relates to Firefox vs other browsers when using the browser based version of XCAS. I've ran XCAS web on Firefox, Chrome, Edge, Safari, etc (on Windows and macOS) and a few different iOS browsers and XCAS appears to run just as well across all platforms both in terms of speed and symbolic rendering. Why is it that you only reccommend Firefox?
All of my hardware is from the last 12 months so maybe it's simply a case that recent well spec'd hardware is fast enough that less performant browsers don't exibit their disadvantage.