December 24, 2009.

The basic idea is to clone an HP48 in software.  However, I want to make
a number of simplifying assumptions.  I recently used an HP48 emulator
and discovered that the design is not nearly as clean as I remembered.
For one, it takes massive advantage of its huge 49 key keyboard.  Only
six of the buttons are soft buttons -- the rest are assigned up to 4
functions each (orange shift, blue shift, alpha, and regular).  So a lot
of things like "store into a variable" are designed to be keyboard
functions, rather than uniform interactions with a soft menuing system.
Also, it has a number of forms to fill out, like graphing parameters
(range, parameter type, etc) and configuration.  The forms totally break
the stack model, and basically I don't like them.

Also, to the greatest extent possible, I want this thing to be
programmable on the run.

So basically it will diverge tremendously from the hp48, so I am calling
it soft84 instead of soft48.

So I figure it's a matter of collecting a ton of design goals and trying
to find their intersection.  It has been frustrating because there are so
many design elements to consider at once and I really want to make
something extraordinarily general that can satisfy _all_ of them.
  * extremely uniform forth environment with a simple kernel
  * data types which can reside on stack:
    - arbitrary precision floating point, with units
    - precise arbitrary length integers
    - functions
    - strings
    - lists of elements
  * math functions:
    - general: abs, ceil, floor, +, -, *, /, %, exp, log, pow, XXX
    - trig: sin, cos, tan, asin, acos, atan, hyperbolic versions, XXX
    - calculus: derivative, integral?
  * stack functions:
    - drop, dup, swap, rot, pick
  * math state: display base (2, 8, 10, 16), angle (1.0, 6.28, 360 units/circle))
  * display: at least 5 lines of text, as many buttons as possible
  * buttons to start entry:
    - string begin (end at ENTER)
    - precise base number begin
    - code begin (
    entry modes: string, precise 2/8/10/16-base number, list edit, code edit

Okay, here's one screen layout possibility.  Landscape mode.  We can have
an 8x8 grid of 64 buttons that are fairly finger-friendly and could
accomodate up to 4-letter labels.  Beside that we have 13 lines of text
which can each accomodate about 20 characters of text (which is plenty).

We'll probably want to mix that up a bit, though.  Especially, some
buttons may need to be larger.  For example, the main number buttons
should be extra large.  Also it may be useful for some buttons to display
the value currently associated with them, so they would need to be wider
than 4 characters (more like 10).  Also, 13 lines is not necessary, so we
can shorten that somewhat.

I am torn about function input.  Functions are first-class objects, and
are essentially little more than a list of words and constants.  It may
be valuable to know how a function interacts with the stack, so that you
can apply functions to eachother.  For example, if you have a function
sitting on the stack that returns one argument, and you apply a function
to it that takes two arguments, it becomes a function which uses both the
function from the stack as well as the previous entry on the stack.

Or maybe there is no need to be aware (a-priori) of the number of
arguments a function needs.  Applying a function to a function would just
always generate a new function ending in the application.

Um, might as well steal some syntax from HP48.  Functions are
  << word word word >>
Lists are
  [ elem elem elem ]
It seems that HP48 had a concept that would implement arguments in a
fairly reasonable fashion:
  << formal1 formal2 -> formal1 2 ^ formal2 + >>


In this way, you could start a function by putting an empty function
(essentially bottom) on the stack, and then type 2 ^ +, and it would
essentially add them into that function and you would get << 2 ^ + >>.

We have a few different screen modes:
  * graph viewing:
    - graph takes most of the screen
    - ability to select ranges within the graph, zoom in, zoom out,
    compute numerical integral, view value under cursor, exit graph mode
  * stack mode:
    - enter numbers, enter edit modes for top of stack, enter regular
    functions, stack manipulation, tools to edit softmenu (store/recall),
    softmenu driven by: math, programming, etc categories
    - edit softmenus, exit from stack mode if nested
  * editing a list
    - enter numbers, strings, arrow keys, delete, softmenu driven by: edit (special editing functionality of ...?), units, and ???
    - special recognition of matrices??
  * editing code
    - enter numbers, units, strings, regular functions, full stack mode
    soft-menu plus edit

Buttons belong in several functional groups:
  * numeric entry - 0-9, ., E, +/-
    - stack mode, 
    - bigger buttons (except for 3 modifiers in corner):
      7   8   9
      4   5   6
      1   2   3
      0   . E -
  * basic math: + - * / 
    - fixed, smaller
  * basic stack interaction: enter, drop (can double as backspace)
  * mode set buttons:
    - insert empty function
    - string start

So many potential modes!

Soft menu should work like ... basically we should always have some keys
on the screen which essentially are the root of the softmenu tree, and
they depend on the mode.  But then we should always have the softmenu on
screen, and also something to navigate it by (to return to the previous
level, for example).


December 25, 2009.

I keep on having trouble determining if something should be accessible
through the softmenu or if it should have its own button, and if it
should have its own button whether it should be permanently visible in
most modes or just the stack mode, or what.

The disadvantage to putting things in the softmenu is that they require
navigation through a hierarchy.  Even if that navigation is cheap, it
interferes with the existing navigation.  For example, consider that the
user is working on stuff in his user/project1/test "directory", and he
wants to overwrite one of the members with the value on the stack.  On
the HP48 he would push the STO button then click on the soft button
corresponding to the member.  If STO is under the PRG menu, though, then
he must press, at a minimum, PRG STO USER member, and that's assuming
that USER returns to the directory he'd started in.

Now that I know I can comfortably fit at least 40 buttons on the screen,
I am very reluctant to do that.  

Oh and complex could be a two-item list or, possibly, a function
dependent on i?  That sounds preposterous.  But it's enough that I'm
going to ignore it for the moment and if I have to retrofit in a new
first-class type, so be it.

Anyways, back to buttons.  I think perhaps the basic programming keys
should be on almost every screen.  So I'm envisioning a softmenu area, a
basic entry/math area, and that set of operations which by their nature
always reference a softmenu button (such as STO).  Basically, everything
that the HP puts on a shifted mode we will instead visit through
softmenus, such as graph, detailed memory management (such as the
"LIBRARY" button), mode set, unusual math functions, etc.

Um, let's go ahead and list the HP buttons:
  row 1: 6x soft buttons
    -> our softmenu
  row 2: math(print), prg(io), cst(modes), var(memory), up(library), next(prev)
    -> math and prog and user (replaces cst and var) are top-level softmenus,
       print/io/modes/memory/libray are burried in the softmenus or absent,
       next/prev are part of softmenu navigation,
       arrow keys
  row 3: '(up,home) sto(def,rcl) eval(->num,undo), left(picture), down(review), right(swap)
    -> ' is for entering strings (just enter alpha into stack),
       up/home are part of softmenu navigation,
       sto/def/rcl are clumsy softbutton creation mechanism,
       eval maybe under prog menu (apply top of stack function),
       ->num/undo/picture/review are irrelevant or can be burried
       swap is stack manipulation
  row 4: sin(asin,delta), cos(acos,integral), tan(atan,sum),
         sqrt(sqr,y^(1/x)), y^x(10^x,log10), 1/x(e^x,ln)
    -> all of these go under math
  row 5: enter(equation,matrix), +-(edit,visit), eex(2D,3D), del(purge), back(drop,clr)
    -> enter is basic stack manip (dup if no text),
       +-/eex are basic numeric entry,
       need way to start and edit list, function,
       2D/3D are mode (really, just number of entries in list),
       back/drop are a single backspace button (drop if no text),
       del/purge/clr must be file manip
  row 6: alpha(user,entry), 7(solve), 8(plot), 9(symbolic), /((),#)
    -> alpha is needed to start a string
       user/entry/solve/=plot/symbolic are irrelevant or can be soft menu
       numeric entry,
       basic math
  row 7: orange shift, 4(time), 5(stat), 6(units), *([],_)
    -> [] starts a list,
       orange shift/time/stat/units/_ are irrelevant or can be soft menu
  row 8: blue shift, 1(rad/polar), 2(stack,arg), 3(cmd,menu), -(<<>>, "")
    -> <<>> starts code,
       "" starts another kind of string??,
       blue shift/rad/polar/stack/arg/cmd/menu are irrelevant or can be soft
  row 9: ON(cont,off) 0(=,->), .(,,ret), SPC(pi,angle), +({},::}
    -> {} is a redundant list starter (lists and arrays are different in HP),
       most of these are a mystery to me, but I will need . and, at least
       in some modes, space (next element).

So it's a matter of listing these in buttons and then in a softmenu hierarchy.
   4 arrow keys: up/down/left/right
  13 numeric entry: 0-9, ., EEX, +-, 
     file manip: ??? XXX
   3 stack manip: enter(dup), backspace(drop), ?(swap)
 ?14 soft menus (7x2 grid...top-level on one side, bottom on the other,
	with status display somewhere and next/prev/up)
   4 basic math: +, -, *, /
   3 begin entry: string, list, function

So I'm seeing something like 50 buttons, which is on the upper end.  But
perhaps I will think of a clever way to do file manip???

Anyways, in certain edit modes, the stack manip and basic math and begin
entry buttons will be replaced with special things like end entry, step
to next entry (space), and in string mode the number keys will even be
replaced with a little shitty on screen keyboard) So it would be nice if
they were visually grouped.

For the file manipulation, I could make a single button that hides
others.  In fact, it could be a regular top-level softmenu, with the
special behavior that it saves the previous softmenu state and restores
it when an action is selected.

So I'm beginning to see the layout, for stack mode:

*             *          *   *
  stack manip    toplev    so
*             *  el sof    ft
                 t butt    me
    numeric      ons       nu
               *   * 
                 be *    *
*             *  gi   nxt  
  basic  math    n    prv
*             *     *    *    *

I'm seeing a need for only a few top-level buttons.  So basically the
stack manip/numeric/basic math take up half of the available area, all
with fairly oversized buttons.  Basically I'm working on a 6x6 grid now,
with some smaller regions.

Anyways, as you'll note, I discard the arrow keys in stack mode.
Basically, they'd let you walk the stack to select older entries, so you
can do that instead just by clicking on the entry, and that would DUP it
to the top of the stack.  If you click on the top of the stack, it opens
the appropriate editor for that entry.  Note that this means the stack
display itself needs a way to scroll up/down for super-sized stacks
(perhaps my typical skinny UP/up/dn/DN bar along the right side of it).
Also, the stack display needs a line at the top to indicate the current
softmenu path - clicking on it goes to the clicked path.

I think I'll discard next/prev as well, just changing the bottom button
of the soft menu into 'next' as needed. Perhaps it can be a 7-button
softmenu?  So really we need to be careful that no softmenu is longer
than about 12, maybe 18 entries at the most, for convenient navigation.

Top-level buttons are a real open book but I'm looking at fewer than 6 of
them, perhaps as few as 3.

So anyways I'm seeing this basic layout.
  13 numeric entry: 0-9, ., EEX, +-, 
   3 stack manip: enter(dup), backspace(drop), ?(swap)
   4 basic math: +, -, *, /
   4 begin entry: string, list, function, precise
   ?4? top-level buttons...
   7 soft buttons

So 34-37 buttons, so we have a bunch of choices around a 6x6 grid.
of choices.

Anyways, top-level buttons are a real open book.  I need at least MATH,
USER, and MISC.  I should work out a bit of this structure, at least
second-level...
  math/trig/{,a}{sin,cos,tan,d->r,r->d,pi}{,h}
  math/??/{1/x,sqrt,sqr,nroot,log_10,log_e,log_n,exp_10,exp_e,pow}
       (be clever because 1/x,pow,log_n,e is sufficient)
  math/calculus/???
  math/vector/{abs,dot,cross,v->,->v2,->v3} (from HP)
  math/matrix/all sorts of stuff on HP!
  math/list/{delta-list,sigma-list,pi-list,sort,revli,add} (from HP,
      maybe should be under program?)
  math/hyp/{expm,lnp1} (HP puts sinh/etc here too, what are these extras?)
  math/real/{mod,abs,sign,mant,xpon,floor,ceil} (perhaps s/real/int/?)
  math/base/{real->precise,precise->real} (maybe these should go under prog?)
  math/base/logic/{and,or,xor,not}
  math/base/bit/{shl,ashr,shr}
  math/plot/???
  math/prob/{comb,perm,!,rand,rdz,utpc,utpf,utpn,utpt,ndist}
  math/fft???
  math/complex/{real,imaginary,complex->real,real->complex,abs,arg,sign,neg,conj}
  prog/stack/{dup,drop,swap,rotn,pick,etc}
  prog/control/{if,then,else,for,next,while,etc}
  prog/compare/{eq,ne,gt,lt,ge,le,min,max}
  prog/dictionary/{a-z}/everything (expose our rpn dictionary)
  prog/type/conversion helpers
  prog/file/{create,delete,store,recall,mkdir,chdir}
             (these take a string name of variable, non-interactive)
  prog/list/???
  prog/io/???
  prog/debugger/???
  misc/mode/base (select 2,8,10,16?)
  misc/mode/vector (rect, cyl, spher)
  misc/mode/angle (degree, radian, unitary)
  misc/mode/prec (number of bits in mantissa of floats?)
  misc/file/{create,delete,store,recall,mkdir,nameof,getcwd}
  misc/quit

That seems pretty managable.  Misc could stand to pick up a bunch of
cruft.

It seems important now to turn our eye to the forth that underlies it.  I
think basically it will use a hierarchical dictionary.  Um, so as not to
confuse ourselves, though, its paths are somewhat different from the
softmenu paths.  So let's use a different syntax, : instead of /.  So it
has basically two hierarchies:
   :builtin:{sin,cos,abs,plus,minus,etc}
   :user:mirrors the user softmenu directory
There is no hierarchy under builtin, and the language hierarchy for
:user: is the definitive version.  The builtin dictionary is what is
displayed by prog/dictionary.  It is every function which is defined in
C.  These functions will simply be basic stack functions, for the most
part.  Variables (in the code editor) are displayed just by their name
(without hierarchy position) unless they are in the :user hierarchy and
are not above the current position in the :user hierarchy.  But in memory
the reference will be absolute, um, at all times (i.e., even in the
conf/state file, I suppose).

Then the softmenus will be of two types.  user/ is simply a display of
the language :user: hierarchy, and has no separate information.  The
other softmenus, though, will be defined in a file.  They will be one of
two types: call C code, or a value.  If they are C code, clicking on them
calls a C function, and this entry cannot be changed by store/recall
(though it will still be in the config/state file...the C code will be
identified by string name instead of by function pointer, so we have some
level of abstraction at which it would be convenient to remap everything
to a different UI or something).  If they are an executable value,
clicking them causes the code to execute (though it can be recalled
without execution using file manipulation).  If they are a data value,
clicking them causes that value to be pushed on the stack.  In addition,
if they are a data value, we should attempt to display the value in the
softbutton, for convenience.

I am having trouble deciding how to deal with things like modes.  For
example, base ideally should have a :builtin:get_base, :builtin:set_base.
So we could make /misc/mode/base/2 which holds << 2 set_base >>.  No
problemo.  But then, how do we display the current base?  Do we have a
'get' function?  Do we just display it in small font in the corner all of
the time?  How about to set arbitrary base, do we have a 'set' function?

Perhaps, in fact, all of the softmenu things which correspond to language
values should be stored in the language as the definitive version, and
they should only appear in the special softmenu definitions if they call
C calls.

C calls need to be more intelligent than language builtins.  In general
they need to be very mode-sensitive, especially when it comes to edit
modes.  it may even pay to have them aware of whether they are called as
arguments to store/recall?

State information to save across invocations seems like it would be three
sections:
   * stack state (just a dump of the values on the stack)
   * non-user softmenu (and key?) assignments in special language (just
     for C calls?)
   * user value definitions (probably something simple like ::name: value ;)

So anyways basically you can see my hope is to be able to implement a ton
of core language builtins, and a bunch of "C calls" for things like
interactive file manipulation, keyboard handlers, and so on, a GUI, a
stack displayer, several magical editing/graphing modes...and then just
fill in the conf file to attach the "C calls" to buttons/softmenus, and
interactively build up the softmenus from there (for example making all
of the variants of exp/sqrt/etc based on :builtin:pow).

Oh also, the softmenus need to have an explicit order, so we need both a
user interface to manage this (perhaps a function which takes a directory
name and a list of the files in it) and a config file hack to make it so
(perhaps just store the list in :path:.sorted).

This is starting to seem kind of tangible.


December 26, 2009.

Okay, here's a proposed syntax for the config file:

# stack state
<num_values>
<value1>
...
<valuen>

# language assigments
:keys:1 !one ;	# assign "one" key to evaluate to a C call of "one"
:keys:+ add ;
:math:exp:exp << con_e swap pow >>

# end of config

So basically, there is only the language assignments, and they define
everything about softmenus and everything except naming of the buttons.
The special C call behavior is accomplished by inspecting the value to
see if it holds one of these magic call things.  And for brevity,
anything that is in builtin is not spelled out in the config file, but
otherwise it needs the full path specified (though the display may take
some shortcuts).

Um, basic lex diagram for the language:
   digit or period or minus digit or minus period -> start of a number
   # space -> comment
   # -> start of an integer (base 10 assumed in the file)
   ! alpha -> start of C call
   other -> start of symbol

So for example + is just a symbol.

And let's just define the language to be lower-case.  In the future maybe
we'll make the display be uppercase...

So I need to think a little more about math.  I think I'll use MPC, which
is basically MP+MPFR+complex (so, an arbitrary precision complex floating
point library).  In fact, I think I will have every floating point number
be complex.  So I need to add a key (probably in the math softmenu) to
emit, I suppose, 0+1*i.  It is tempting to just treat complex as vectors,
but I think that will backfire at some indeterminate point in the future.

Note that this is going to be quite slow, just like "real" RPL.  But
hopefully today's megaprocessors will make up for it.  The last time I
made a forth interpretter using this methodology, I knew it would be
slow, and I was shocked to see that it took on the order of 0.2 seconds
to process a keypress.  But on the other hand, it was an entire IRC
client, including line editing, implemented in this forth.  Everything on
the level of line editing in this one will be implemented in C.  So it
should only matter at all when it is graphing.

I find I need type promotion rules.  I guess for a function that expects
a matched pair of arguments...if either element matches a member in this
table (searched from top to bottom) then the other is converted to the
same type:
   function, list, string, float, integer, list

And the semicolon operator is totally unecessary since there is only one
value per variable, so the list is just
   :path:var value

In fact, a function can be exactly the same format as a list, if we allow
references to variables (calls of that variable) to appear in lists, or
at least make the data types support it.  I guess it is still useful to
strongly differentiate between the two, though, because so many UI
elements will do so.

Apparently it's most convenient to use MPC's complex format, which is
  (1.2 3.4)

But for display I am using 1.2+i*3.4

Well, I've been hacking on it for a while, and I seem to be inclined to
work on the language functions now.  I've now got most of the bookkeeping
code and data structures in place just for saving and restoring objects
and so on.  So I guess it's time to make a roadmap:

  - make interpretter for language
  - make stack mechanics for language (push, pop, convenient type
    conversion, etc.)
  - register some builtins for the language
  - invent the basic editor and entry interfaces
  - create the front-end
  - use it

For the most part, this code has been pretty mindless.  Most especially,
I think it will be very quick and mindless to fill in all of the wrappers
to GMP/MPZ (arbitrary int) and MPC (arbitrary complex).

The config file does turn out a bit more sensibly:
[ stack is a list ]
:user:var << this is a definition with a value >>
!user:var defn_is_C_call

And now I'm calling "C calls" "key_handlers" instead.  For the moment
they are just a relatively bare function pointer, but in the future they
could provide all sorts of information to a potential caller.

Anyways, the only part that has been making me nervous is the
editor/entry interfaces.  I am leaning towards having a general nestable
entry mode that takes up just the bottom of the stack display.  It will
be where keyboard input goes, always.  But if you are in an edit mode of
an existing object, then it shows up and you can select members within
that object by clicking on them.  For functions and lists, especially,
you select by value.  Anyways, a selected object could be deleted, insert
new value after (default behavior if you just start typing), edit value
under cursor.  For editing a string, it is a little less clear how that
should work, and also it is not clear how to be able to enter variable
names in a function to call undeclared functions...I think maybe I just
shouldn't, all functions should be declared prior to being added by the
UI.


December 27, 2009.

Okay, I filled in a bunch of basic math builtins.  They're going in easy,
and I'm confident my data types are pretty reasonable, though there sure
is a lot of potential copying going around and also memory allocator
load (might even be worthwhile to put value_recs in a pool allocator).
So that's reassuring.  Now it is time for another todo list:
  - invent the basic editor and entry interfaces (implement keyhandlers)
  - create the front-end
  - flush out builtins (especially syntax builtins like IF/THEN/ELSE)
  - use it

So I'm really quite close to working prototype, but I don't really _want_
to invent the entry interface.

Basically, plot mode is the only mode that can't have an entry box.
  * stack mode - need to be able to input numbers, begin other modes, and
    execute ops and softmenu items
  * number entry mode - entry box takes up bottom line of screen; can
    navigate around current number by clicking in it; any executable
    will generally finish entering the number
  * list entry mode - entry box takes up entire screen; need to be able to
    input numbers, begin other modes (they nest), move around the list,
    begin/end sublists, and nest into stack mode to fill in the current
    cell; minimal need for softmenus, ops, etc.
  * function entry mode - entry box takes up entire screen; need to be
    able to input numbers, begin other modes (they nest), move around the
    function, nest into stack mode to fill in the current cell; ops and
    softmenu items are inserted into the function
  * string entry mode - will work same as number entry mode, except
    needs to provide access to buttons that switch out the number entry
    area for upper/lower alpha, symbols, etc, and end string

So I suppose really there are three non-plot modes: stack/list/function,
and all three of those can have an entry line at the bottom.  And there
should be an "edit" command that takes a value_rec and begins the
appropriate entry mode.

...

I have gone away and come back some hours (and drinks) later and I am
little clearer on the subject.

One thing I keep coming back to is whether the UI needs to know the type
of the final value.  I guess it needs to know function vs. anything
else, to decide how the softmenu buttons should act (to insert calls
into a function).  But other than that, maybe I should have a general
text input that can take [ and << as regular tokens?

I really don't want that to be the answer, but maybe it is.  I suppose I
should use the HP48 emulator for some meditation on the subject.

But maybe the natural way to categorize them is string/number vs.
list/function.  Since a list/function is mostly made up of
string/numbers, and never vice versa, it is sensible and natural that
they should be input in different ways.  Then it's just a question of
how deeply to distinguish list from function.

I'm toying with the idea of discarding the 'begin' buttons, and putting
them in a softmenu, perhaps 'misc:begin:*' with members like int, string,
list, function.  There should also be a way (probably through softmenus)
of inserting 'i' into the middle of an numeric entry, and there needs to
be something that can read that form.

Okay, the HP48 has a general edit mode that can edit strings.  It makes
it slightly easy to delete and insert whole tokens within the strings.
It is data-agnostic.  It will let you fuck up the syntax and it will let
you insert function calls into lists and things like that.  If it has a
syntax error when you exit the edit mode, it won't let you exit the edit
mode.  Which seems pretty shoddy to me, really.   If you enter matrix
mode then it is nearly exactly as I imagine my list mode to be.  If you
begin entering something, it appears in a separate entry area and only
getns copied into the matrix display when you are done editing it.
Roughly like some spreadsheets.  So I think that is probably the way to
go.

Now it just comes down to buttons.  If I abandon the begin hard buttons
then I can have a really uncluttered display, using nearly half my
button area just for softmenus, which is especially spare if we only have
4 top-level softmenus.

But that doesn't leave a super large area in which to implement the
scene-specific buttons.  For example, if the stack mode is nested within
another edit mode, for which it is producing a value, it needs to have
an 'exit' button fairly prominently, and I don't think ENTER can play
that duty.  For list/function editing, we need access to a special
softmenu of edit functions (forward delete, ?done?, insert from stack,
??).  Maybe edit ones could go where the basic math (+-*/) goes, and the
exit button could be an empty space by the top level softmenu buttons.

So just to gloat, our display is roughly down to:
  * stack manip * top**soft
  * numeric...  * lev**menu
  *     ..      *  el**  ..
  *     ..      *    **  ..
  *     ..      *     *  ..
  * basic math  *     *  ..

For a total of, um, 31 buttons, or 32 with the exit button.  Very
luxurious.

I'm tempted to put the top-level buttons on the right side of the stack
display.  

So that is kind of gelling together in my head.  But I also have need
for a completely unusual display object.  I think the regular text
display can be done through a regular textlist without any difficulty.  The
number/string entry might even fit within a textlist.  But I need to be
able to really pack values in (several per line).  It would be nice if I
had a high-level type that I could just insert value_recs into, and it
would implement linebreaking them (in two modes, one for stack mode and
one for list/function editing) and selection of them.  And it would need
to allow also having, say, the lower line (or two?) of its output
commandeered by number/string entry.

All this time I've been thinking of how to hack textlist to accomplish
this, but it now occurs to me that the best way to accomplish this is to
make a totally new single-purpose widget.  I'll need the ability to draw
colored rectangles and to render fonts.  So I think the existing
ggl_drawable class should fit the bill.  It would accept a linklist of
value_recs that it would allow you to view (with scrolling if
necessary), and to select (highlighting whole value_recs with inverse
video).  It would also be responsible for tracking the entry buffer at
the bottom.  Entry basically needs entry_char(), entry_stop().
entry_char() will be called whenever a new character is available for the
entry buffer and will optionally start the entry mode.  entry_stop()
would end the entry, like by hitting ENTER.  An entry can be navigated
about by clicking inside of it, and the cursor will be an inverse video
square.


December 28, 2009.

I have re-read yesterday's entry and I'm still convinced that it is correct.
So let's break down the remaining tasks somewhat more thoroughly:
  - multidisp.c - the general multipurpose display of values and entry buffer
  - softbutton.c - keyhandlers to implement softbutton menu, etc.
  - gui.c - lcars initialization, layout, event disptach handlers
  - keyhandlers.c - general keyhandlers
Um, is that it?  I guess there's plotting, but I'm going to leave that
for last.  I think if the programming language is any good, I should
actually be able to write plot mostly in its language.  And then there
are some language features like named local variables to consider as
well.

Anyways, multidisp.c.  It's a ggl widget that tracks two different lists.
One is a list of value_recs, which will be either the stack or the
currently edited function/list.  The other is a list of characters (a
string) that is the current entry buffer.

It needs a few functions to interact with the list.  Basically, it needs
to be able to announce that it has updated, to set stack mode, to insert
an entry at the current cursor pos, to delete the current entry, to
scroll the list (the scroll buttons should be outside of this widget).

It needs to interact with the entry buffer as well.  Insert at current
cursor pos, clear entry, finish entry, backspace.


December 30, 2009.

Maybe value_recs should be garbage collected?  It would certainly be easy
enough to pull off, and they certainly do involve a ton of malloc/free...

Yay!  At only 2811 lines of code, we have the first runnable version.
And with only a little bit of hacking, it works.  That is to say,
multidisp is fairly well-behaved in stack mode, and enough of the
language works for basic key bindings and math to operate.

Now there are a bunch of things opening up on the todo list:
  - status bar (at least directory part of it)
  - implement sto keyhandler
  - flush out menus
  - finish implementing edit modes (multidisp will do them, so basically
    i need a way to start them, and the appropriate soft menu for them, and 
    special inputs like for strings and so on)
  - make 'quit' key handler, and save config even on forced quit
  - modality?
  - visit all of the XXXs
  - units
  - programming functionality
  - plotting

But the real #1 thing is: need to build GMP, MPFR, and MPC for the n810!


January 1, 2010.

And it is really coming together, but it is becoming more obvious how
many things are pending.


January 2, 2010.

Starting to really work on edit, and it is going well.  But I realized
that I really need a general nestability of gui modes.  When we are
editing a list and we decide to edit a member which is itself a list, we
need to go into list edit mode recursively.  Plus we need to be able to
enter stack mode.

I want to minimize the amount of state saved as we recur into these
modes, so let's try to enumerate everything that is needed:
   o) stack: nothing needs to be saved, just note that it's a stack
   o) edit: value being edited (do not allow multiple edits of same
      variable!), pointer to current position, startrows, 
   o) plot: do not allow nesting
    
Oh man and it is working, nested edit modes!


January 3, 2010.

Okay!  Little todo list items disappear pretty quickly.  For the ordered
menus, I'm just going to make symbols be automatically added to the end
of the directory, so they will be in the order of the config file
(instead of inverse of it), and saving them will be in the proper order.
Now I just need to implement a builtin I'm calling "order" which will
take a list of variable names (vk_calls), and order the directory
accordingly.  So the upshot is there will be no separate list of the
order of the directories.  Also, whenever something is appended to the
builtins directory, it can automatically also be inserted into the
appropriate spot in prog:dictionary.

And I think I've decided to make a new hardbutton to go with "sto",
called "ref." It will generate a vk_call for whatever button you next
push, and put that on the stack.

Okay, so with the ref button it is pretty convenient to do most file
accesses.  However, one thing is still rather clunky, and that is
creating a new file.  Here is the current process:
   user->mydir (navigate to target)
   123 (enter value)
   misc->edit
   "
   1aA! (enter name)
   enter
   misc->files
   store

The only slick part is that you do not need to renavigate to the target,
as a string used as a filename for store will be in "curr_user_dir" by
default, if no explicit path is specified.

The particularly clunky part is that you must navigate to misc->edit in
order to enter a string, which is just really unfortunate in practice,
especially if you accidentally navigate away from it in the wrong
keyboard mode.

Perhaps the status bar should show something of the keyboard state, and
clicking on it should change it?  Then there is just the question of the
" and the :, both of which could be made more easily accessible in the
alpha mode?

Indeed, that's a pretty usable solution.  So now it is fairly convenient
to do all of the basic editing tasks, as well as most of the basic
directory maintenance tasks.  List and function input and application
work.  It's really getting pretty exciting!


January 5, 2010.

For some reason I was a little intimidated when I considered the quit
functionality.  It turns out that detecting forced quits on the n810 is
trivial, though, as it sends a WM_CLIENT_DELETE message that was even
being received (it just destroys the window, rather than invoking quit,
though).  So fixing that upstream problem in gglcars made it trivial.
Literally 9 lines of code and it's done, and working amiably.  And it
gave me a chance to test the config file saving, as well as adding the
misc:quit button without editing the config file (i.e., using its own
GUI).  I am pleased on all accounts, with the detail that saving complex
numbers is ugly, generating something like
"(5.00000000000000000e-1 0)" instead of "0.5".

There, that wasn't so bad.  


January 6, 2010.

Well, my todo list indicates that figuring out units is next on my
agenda, though probably fixing up the msic menu should be higher.

Units are fairly simple.  Each floating point number will have a pointer
to a linklist of units elements, each one of which specifies a unit, a
exponent, and a power.  The unit is something like grams or pounds.
The exponent is 3 for kilo (k), 6 for mega (M), -3 for mili (m), etc.
The exponent will always be 0 unless the unit indicates it is SI.  The
power is the power of the unit.  Square feet, for example, would have a
power of 2.  One sticky part is that the power may be a decimal (for
example to the half power if you take a square root of something), so we
really need it to be an mpfr.  I think I will have it be an mpfr, but of
a fixed and relatively low precision, and it will display very few digits
of it when displaying it, and it might even arbitrarily clamp it, or just
display poorly if it exceeds a reasonable range.

On the HP48, most operations are not happy about units.  For example,
sin() and ln() cannot be applied to united numbers.  Units may not appear
in the y of an x^y, but may appear in the x and have y applied to their
power if so.  + and - need to have units of the same kind.  * can work
with any units and just merges the lists.  / can work with any units and
merges with the inverse.  These behaviors all seem rational to me.

The user interface on the HP48 is kind of interesting.  If you just hit
the unit soft button for grams then it is like "1_g *", so it merges the
unit with your unit list.  If you hit one shift and grams then it is like
"0_g +" (converts to grams), and the other shift mode does "1_g /".

So I need functions to convert units, to merge units (also by inverse),
to apply a power to units, to clear units, and to normalize a united
number (i.e., change 1000g into 1kg), and of course display functions and
so on in addbuf_value() etc.

The real question is the data structure to define units themselves.  It
needs to provide at a minimum the name of the unit, a boolean indicating
if the unit is SI or not, and a definition of the unit in terms of at
least one other unit (perhaps absent for some SI units).

Then there is the matter of a heuristic for converting from one unit to
another, and a heuristic for making units sane (i.e., ft*m should
probably be m^2 or something?), which maybe should be an explicit step...
Maybe something that generally converts towards SI and something else
that generally converts towards english?


January 7, 2010.

Okay, there are two units things on my mind.  One is conversion, and the
other is specification.

I'm pretty sure conversion will work out pretty easily, but I'm a little
torn about it mostly because of how it relates to storage.  Units
group into roughly four categories: length, time, weight, and temperature.
Volume is just length^3, for example.  So to determine if it is
possible to convert one unit to another, it is handy to separate the
units into their components.  For example, a Newton is kg*m/s^2.  In
other words, weight^1, length^1, time^-2.  So if we have something in
random units to convert to Newtons, we just need to go through the
list and convert the weights, then the lengths, then the times, and
anything left-over means error.

But all of this is dancing around the way to specify units.  I very
much want the units to be specified in "userland" because I want it to
be seamless to add new units.  As part of this, I want the definitions
to be very loose.  The unit should be defined solely in terms of
another unit (like 1_N = 1_kg*m/s^2 or 1_L = 0.001_m^3).  The easy
conversion approach for complex units that I described above does rely
on a determination of whether a unit is complex or not.  I really do
not want to specify that a unit is a length, nor do I want to specify
explicitly that a unit belongs to a group of units that can be
converted by straightforward constant multiplies.

The rational place to specify units would be directly in the
misc:units menu.  The trouble is that I have a few different usages
of them.  I guess it can be reduced to: looking up the unit as part of
computing compatibility/conversions, and applying the unit to the
stack.  If the program simply magically "knows" about units, then for
example, we'd have a definition like:
  :misc:units:g << 1_g * >>
Another alternative is the language would recognize that :misc:units
is magic, and then I would define it like:
  :misc:units:g [ "gram" "g" builtin:unit_si 0 ]
  :misc:units:lb [ "pound" "lb" builtin:unit_english 123_g ]
So basically a list with long name, short name, SI/english/misc, and an
equivalence in another unit, or 0 if it is a base unit.

misc:units would be special in that the language would pre-read it
(um, and re-read it if changes?), and also in that a keypress on
something in misc:units would have the effect of merging the unit with
top of stack, instead of trying to do something with that list.

So I think as elements are added to the dictionary, if they are a unit,
they should be directed through something that accumulates them in a data
structure.  The data structure should be organized as a list of lists.

Um, I'm going to declare by fiat that you cannot reference a unit which
hasn't been defined yet?  This produces a number of infelicities, though,
as it imposes ordering restrictions on our config file and our config
file is already ordering specific.  But it is not a serious ordering
restriction, it basically means that base units must come first in the
menu.  Whatever unit I decide to use as base surely does deserve this
honor.  Worst case it puts me in an awkward situation of interleaving
definitions for different sub-menus of misc:units.

Anyways at the top there is a list of unit classes, and then each unit
class contatins a list of all equivalent units.  When a unit is added,
if its equivalence is specified in terms of a single other unit then
it is added to whatever unit class that is.  If it is specified in
terms of a compound unit expression or a base unit, it gets its own
class.  So the first member of a class is either the base unit or the
base compound unit.  Whenever we add something to a unit class, we
record its equivalence in terms only of this anchor unit.  For the
anchor unit of a compound type, the equivalence is stored in relation
to the anchor units of its member types.  

So to convert anything, we have a process.  You know, it can be
optimized a bit, but it ultimately comes down to: convert to anchor
units, convert to destination units.  To check if two different units
are the same, we just convert the units to anchor units and then
compare the anchor unit sets.

You know, I think this idea is fleshed out.  Engage!

...

I've been working on implementing the units for a few hours now and it's
coming along kind of swimingly.  However, I keep getting this sinking
feeling every time I think of tracking the power in any sort of floating
point number.  It is neat that the HP48 will allow you to take a
sqrt(4_s) and it returns 2_s^0.5.  But it is not useful, and it is really
awful to think of using it in practice.  So I'm going to just drop the
idea -- power is now an int!  A nice sane int!!


January 8, 2010.

Okay, I got units to "work" (i.e., I can do conversions between meters,
feet, inches, yards, and the squared variants thereof!).  So I made a
list of all of the units on the HP48.  One thing I note is that there are
many units with the same short name, which is bad.  Also, there are many
units that are the same except for prefix (kg vs. g).  Also, there are
just plain a lot of units.  Plus, there are lots of obscure units and
varied units, like US/UK variants.  These I can handle.  What I cannot
handle is degF to degC.  I apparently need, in addition to a conversion
factor, an offset!  I'm sorely tempted to just ignore this problem
indefinitely.


January 9, 2010.

Okay, let's go through my list of issues.

Prefix unit buttons: Let's just go ahead and add these, but I'm going to
go ahead and magic it:
   :misc:units:length:km 1_km
which will be ignored by process_unit_def() but add_unit_to() will
interpret it.  It cannot be
   :misc:units:length:km << 1_km * >>
because the units are applyable to list/function members in the editor
(i.e., in place with no stack interaction).

Prefix unit merges: I think that really when multiplying 1_kg*1_g we
should get 1_kg*g rather than 0.001_kg^2.  It is up to the user to call
normalize if need be.

Unit ambiguity: Whenever a unit is added, if its name begins with a valid
prefix, it should be checked against all the already-entered SI units to
be sure it's not ambiguous.  If an SI unit is added, its name with each
prefix should be checked against all of the others.  Also units should be
checked for others of the exact same name. If they collide by prefix, the
new one is ignored and an error message is pushed onto the stack.  It
would be nice to be able to replace ones that match exactly, but it seems
like to do that it would imply that each time a unit is changed, we could
potentially need to recompute each unit's relation to the others, if the
new unit is a base for any other units.

Zero-power units: Any unit that is basically 1_m^0 should automatically
be removed from the list.  Probably only need to scan for this during
merge_units()?

Disguised zero-power units: Is 1_m/in is a unitless number?
   1_m = 39.3700079_in  => 1 = 39.3700079_in/m (A)
   1_in = 0.0254_m      => 1 = 0.0254_m/in (B)
So we multiply it by A and get 39.3700079_(m*in)/(in*m), or 39.3700079
We multiply it by B and get 0.0254_(m*m)/(in*in).  So I think it is a
unitless number!  Conversions to NULL should therefore succeed when
plausible.

US/UK units: It turns out most of this stems from the fact that in
antiquity there was more than one yardstick which was considered to be
authoritative, and the yardsticks were used so often or so poorly
constructed that they shortened over time!  From this clusterfuck comes a
modern yard, which is used as the basis for all of the other English
length units.  Other yards are only used for historical reasons,
basically surveying.  So if a surveyor or other
purveyor-of-retarded-units decides to use my calculator, he will
appreciate the fact that it is very easy to add custom units.  Hopefully
I will be able to make the same declarations for such stillbirths as
US/UK gallons.

Temperature offsets: Luckily these offsets disappear in compound units.
For example, 1 degF/s is 0.555 degC/s, as it is a rate rather than an
absolute value.  I'm going to make the same claim for contrived units
like degF*s.  So what I need is a simple hack to guarantee that any
conversions between uncompounded temperatures works, and not worry about
anything else.  I'm willing even to accept that it will be thrown off in
the presence of disguised zero power units (i.e., 1_degC*m/in).
unit.  I'm leaning towards making temperatures use a 5-element list, with
the extra element supplying an offset from absolute zero, I suppose with
units (they will be converted to the unit in question).

Would really like a nicer UI for invert unit, and also for convert to
unit.  Convert is currently
   0 unit +
, while invert is currently
   1 unit /
.  Which is only a little of a nuissance, right?  Maybe a general 1/x key
would be convenient, anyways?  Is 1/x more useful than swap?


January 10, 2010.

Okay, now I'm looking at unit normalization.  I think I'm only going to
worry about SI for the moment.  So, normalization will reduce everything
to the base units.  Then it will count the exponent and if it is large
(positive or negative), it will call a function which attempts to reduce
the exponent by changing the exponent on the units.


January 23, 2010.

Okay, cleaning up the infelicities is really making it come together.

The saved stack cannot have units because it is parsed at the very
beginning of the config file.  I think it should be changed to occur at
the end, and any value that occurs without a definition should go on the
stack.


January 24, 2010.

Well, that was an undertaking, but the units are complete (and somewhat
tested)!

The surprising part was how many bugs were in units.c long after I
thought it worked.  Basically, conversion_to_base() was overly naive in
walking the chain of units -> anchor units -> anchor's base units.

Anyways, I decided the idea of changing the config file so the stack
comes after the dictionary appeals to me.  The easy way to do this is
load_config() now recognizes a magic '.' prefix that means "push this
value onto the stack."

The GUI has fairly well geled, and I like it how it is.  However, there
are still cases where it feels like it is obnoxious to walk the menus,
and I still want a "1/x" button and an "edit" button.  And having the
status display for the menu path being the only way to navigate
conveniently to the previous menu (i.e., "cd ..") is counterintuitive.
This is unfortunate, though, because if I move anything around then the
whole thing needs to be rethought, as there are no empty spaces in which
to insert single buttons.

Also, now that it is useful, I want a CLI interface to the same
functionality so I don't have to track down my palmtop to use this.  I
need to sit on that for a while.  Should be a matter of anything you type
is either a literal, a name in the current directory, or a name in
builtin.  And everything you type is executed immediately, or pushed onto
the stack.  Whenever you enter a command the top few of the stack are
displayed, and the current directory is also displayed. There may be a
call for a few additional builtins that aren't needed with the GUI, such
as "display the whole stack" or something.

Anyways, so we're down to those usability questions, and then there is
the design question of the programming language, and the design question
of the plotter.  I think I want to make the programming language first
and then contemplate the possibility of implementing the plotter in the
language.


November 26, 2010.

I want to port the thing to Android.  My Android device has an even
smaller screen than my Maemo device.  The existing soft84 has
approximately a 6x6 grid of buttons (36).  On the Android device there is
a crappy calculator with a 4x4 grid (16), and it is very comfortable to
use.  I could probably compromise at a 5x5 grid (25) but I think a 6x6
grid is simply too tight for my phone's screen.  A 6x6 grid is
approximately as tight as the built-in keyboard.  It is fully usable but
you hit the wrong key often enough that it would be pretty unforgiving in
the calculator, especially if I don't implement an UNDO command.

Android uses Java, but you can call back to C code ("JNI").  So I think I
will put the GUI code in Java and then have that call back to the
existing codebase to implement the calculator functionality.  This means
abandoning GGLCARS but really it's just a grid of square buttons so who
cares.

Android is a pain to compile libraries for, which is unfortunate because
my dependence on mpc/mpfr/gmp is pretty thorough.

Also note that I will need to invent both portrait and landscape layouts.

One obvious simplification I can make right off is to remove the
math/prog/misc/user keys, and just have them appear as the top-level soft
buttons.  I can remove the up/dn stack view scroll keys by making the
stack a native android touch-scrolling list.  I can get rid of clr (clear
entry buffer) by making it the hold-down-backspace functionality.  I can
probably also get rid of the "next" soft button by putting the soft
buttons in a touch-scroll list (!!!).  Can I get rid of ref/sto?
Especially now that navigating the soft-menus to hackily access two at
once is such a nuissance.

I'm tempted to make the thing hyper-modal, so I can really cut down on
button count.

I could hide things like ref/sto behind an Android "menu" button, but
that is essentially the same as using a built-in Android functionality as
a tool for modality.  I would rather make my modality more uniform.

If I have a 5x5 grid, I can lay soft buttons across the top row.  That
leaves 20 buttons remaining.  Basic numeric input (0-9, ., E, +/-)
requires 13 buttons.  And basic stack manipulation really requires at
least two buttons (enter, backspace) and really wants four (swap, edit).
Four basic math operations, plus we really want a fifth (1/x).  Plus we
really want ref/sto (2 more), and it would be nice to have access to most
of the misc:files functionality.  Plus we need the 123/ABC/abc/!@# mode
button, and some sort of status bar indicating mode (stk/edit/??) and
softmenu directory (with the ability to navigate).

So in a 20-button space, that is 19 buttons required with 24 buttons
really needed, and a desire for something like maybe 10 more on top of
that.  In 20 spaces.

On the real HP48 they solved this by being essentially hypermodal.  They
had a blue shift and a red shift, and an alpha shift, so each button had
approximately 4 modes.  The shifted keys had a reliable brief overhead,
and most importantly didn't interfere with the soft-menu state (no need
to navigate back to where you were).

I am very opposed to so much clutter, especially visual clutter, as to
have 4 functions for each key displayed on the screen at once.  I've
already crossed this line with the 123/ABC/abc/!@# modes, which are
pure-entry modes (and do not affect the soft buttons), but they don't
seem onerous to me because they rotate all of the visual cueing.  I don't
want the user to be guessing, though, at what functions are available
behind a mode-shift that exposes operators.

However, Android has this concept of a long-touch (like a right-click or
double-click).  So I can use that as basically a for-free single shift
mode.  Then I just need to display one extra string for some of the
buttons, and have that command be activated when you hold down the key.

If I have 20 slots, that should be pretty ideal for the "required"
category, and then I could have long-touches for the other ones.

Sample portrait 5x5 layout:

  [stack bottom]
  . . .
  [stack top]
  [soft nav??] [status]
  soft1 soft2 soft3 soft4 soft5 ...
    7     8     9   enter back
    4     5     6     +    -
    1     2     3     *    /
    0     .    +/-    E   mode

The "mode" button is the 123/abc/ABC/!@# selector (displaying the next
mode it will enter if pressed).  Accomodating those modes will be a
little tricky.  Maybe they should cause a native Android on-screen
keyboard pop up?

Anyways, some long-touches:
  back -> clear
  +/-  -> 1/x
  plus whatever is needed for softmenu management??

That's not so many, really.  It gives us a lot of flexibility for
whatever shortcuts we need for softmenu navigation/manipulation.  For
example, I could use long-touches on the number buttons to hit the last
opened subdirectory of math/prog/misc/user, just like the old
softbuttons.  And mkdir/erase could also be readily available.

The alpha modes need 27 each (and use 30).  The symbol mode has 32, and
probably needs most of them.  My count of the 5x4 shows that I need to
keep three of them (enter, back, mode) so I can replace 17 of them.
Using long touches, I can then accomodate 34 values without reconfiguring
the GUI at all.  I think that is acceptable, especially since if I really
need to input alpha I will just slide the keyboard out.

Other than layout/key handling, there is one other important GUI
functionality.  It is this thing called "multidisp." multidisp displays a
series of lines.  The top one is optional and represents the current
input buffer ("entry"), while the others are either a stack or something
which is being edited, depending on the mode.  Methods:
   constructor()
   edit(value_rec v) /* set to edit mode, editing v (v must be list or func) */
   pop() /* exit an edit session */
   stack_height() /* return height of stack */
   val_insert(value_rec v) /* insert a value at the current edit cursor */
   add_unit(value_rec unit) /* add a unit to entry or top of stack */
   stack_changed() /* note that stack has changed */
   redraw_for_stack() /* redraw if the stack previously changed */
   value_rec val_delete(int front) /* delete a value from the edit list */
   val_scroll(int dir) /* up/dn buttons, seems to be unneeded now, right? */
   entry_set(char *s) /* set the current value in the entry buffer */
   entry_insert(char c) /* insert at current location in entry buffer */
   entry_delete(int front) /* delete at current location in entry buffer */
   entry_toggle_sign() /* add/remove a "-" at the beginning */
   entry_clear() /* empty entry buffer */
   entry_finish() /* calls input_value() on the current entry buf */
   input_value(value_rec v) /* adds value to TOS or to cursor in edit list */

Right now in general the GUI code is way too coupled to the rest of it.
It needs to be changed to the best extent possible to communicate totally
in strings (and lists of strings, and maybe lists-of-lists-of-strings).
In other words, I need some sort of bridge to carry value_rec across the
C<=>Java boundary, maintaining its separation into individual values and
its association with the source data structure, but providing strings for
everything as well.  The question is basically whether I will devise a
structure which is passed into the Java and has all of the strings in it,
and is then the original value_rec is somehow rebuilt when it leaves the
Java; or should I pass a structure into java which is mostly opaque and
have the Java call into C methods to read out the values as strings and
to do operations like list next and list insert and so on.  Which is
really what seems best to me.

Anyways, the old code was architected assuming that it had a fixed list
of lines that it could display, and then multidisp is responsible for
doing line-breaks and cursor management.  Which is kind of neat in that
apart from assigning strings to these lines, really the only advanced
operation it needs is the ability to determine how many pixels wide a
given string will be.  And it could probably get by simply knowing which
character (entry) or token (edit list) the user clicked on.

For Android it seems like maybe we really should take advantage of the
native widgets (EditText for the entry, and a ListView containing
TextViews for the scroll stack/edit list).  I'm not sure there is any
actual difficulty whatsoever with the entry, assuming it handles multiple
lines well (which I'm pretty sure it does).  It is especially easy
because it is already decoupled -- it communicates with the math part
entirely through strings.

The scroll stack/edit list would be easy to implement by making one
TextView per value, but the trouble is that it is hard to compute the
token which was clicked on.  I see two approaches.  One is to use
Paint.breakText() which can determine the width of a string, and
Paint.FontMetrics which can determine the height.  Then it would be
basically the old game.  The alternative is to make each token a "link"
(<a href=...>) and then register that my main (and only) activity can
handle that URI, but doesn't need to be reloaded.  The latter option
seems like it is the best, except that it is kind of unknown whether
Android will let me trap the URI without starting a new activity from
scratch, which would be a deal-breaker.  The former idea, however,
requires nothing that I don't already understand and trust about Android
(assuming breakText() is capable).


November 27, 2010.

I have an idea for the convenient menu navigation.  I have always been
uneasy with the old math/prog/misc/user buttons that would return to the
previous subdirectory of the specific root directory.  So here's
something much more general.  You essentially have two sets of "soft
button state" (i.e., path), and then there will be a single button which
will swap between them.  That way, for example, to access misc:files
while in user:fred:xyz, you would just navigate to your user path, then
hit this swap-state button and navigate to misc:files, and then
swap-state between them at will.  The question then is what to name it.
"alt"?  So I need "pop", "root", and "alt".


November 25, 2011.

I got a much better Android phone (LG Optimus V).  I am a little more
motivated on this project than I was.  And maybe I have a little bit of a
fresher idea about how to simplify the UI.

It occurs to me that last time I stalled out because I couldn't find an
android function that would return to me the character offset at which I
had clicked.  I basically don't believe that, so I'm going to have to
take that fight back up.  And it is sad to see I really didn't get very
far at UI glue code.  And to make matters worse, I still have to figure
out how to compile gmp/mpf/mpc for Android.  A pain.

But I have a few simplifying ideas for the UI that make me pretty excited.

First, soft menus.  I don't need a "root" button, I think "pop" is
sufficient.  And I don't need "alt".  The whole "alt" problem is the fact
that I need access to store/recall/mkdir/deletevar/editvar kind of
functionality in all of the directories, not just in misc:files.  So I
propose putting them under the Android "menu" key.

Unfortunately, looking back, this is the layout I came up with

  soft1 soft2 soft3 soft4 soft5 ...
    7     8     9   enter back
    4     5     6     +    -
    1     2     3     *    /
    0     .    +/-    E   mode

Which is disappointingly incomplete.  Especially I am upset that it is
missing swap and even a single spot for a "pop".

Playing with the on-screen keyboard, it uses about half of the available
portrait screen and is essentially 10x4.  So it could be 10x5 at as much
screen as I'm willing to give it.  The built-in keyboard is a little
frustrating, though.  Basically, when using my thumb it is difficult to
hit each key centered left-right.  But I usually get at most one key off
in either direction, so if it was half-as-dense horizontally, at 5x5, it
would be just great.

But maybe I can get away with doing some tricks like . +/ E (3 buttons)
taking up the space of two in the grid.  So that'd free up one spot, but
to fill that spot I have swap and menupop.  I don't really want to make
enter any smaller than it is.

Okay, further inspecting the on-screen keyboard, if every key was as big
as the backspace key (which is pretty comfortable for me), it would be
6.6666 wide.  So I could make it one wider without running into too much
difficulty, I think.  In fact, the RPN Calculator that is on it has 6
buttons across and seems perfectly usable in that respect.

Oh and another innovation, instead of "mode", there should just be
"input" which will pop up the regular on-screen keyboard.

And we might be able to get rid of menu pop by using the android OS back
key, but I have a suspicion that actually we won't be able to trap it for
that function (easily).

So that gives me 6x4 for the interesting buttons:

    7     8     9     ...enter...  back
    4     5     6     menpop input swap
    1     2     3     1/x    +     -
    0     .    +/-    E      *     /

And I think that's enough.  Even has room for the extra-wide enter key.

The landscape design can be the exact same thing, but with the buttons
beside the stack instead of below it.


November 26, 2011.

Okay I took care of a lot of lingering issues with the visual appearance
of the basic calculator buttons.  They look pretty good and change color
when they're clicked now.

I don't know what gave me so much difficulty trying to figure out how to
make multidisp work last time.  I simply embedded a vertical LinearLayout
within a ScrollView (just like the soft buttons -- not as efficient as a
ListView if there are a ton of entries but otherwise just fine).  Inside
the LinearLayout, I create TextViews.  The TextViews use a "spannable"
string so that I can indicate highlighting for certain regions within the
string.  And I trap OnTouchEvent so that I can get pixel positioning
information.  Then the cherry on top is that TextView.getLayout()
provides a class that makes it very convenient to determine the text
offset of where the user clicked.

The only infelicity of getLayout is that it returns the character whose
left edge is closest to the cursor point, so basically it rounds up (to
the right).  But since I really only need word-positioning, I think it
can work.  I just need to count the space as belonging to the preceding
word.

The vast majority of coding is still ahead of me, but the wibbly bits of
interfacing with Android are taken care of now.

Finishing the interface (especially multidisp) is pretty tempting.
Unfortunately I think I'll run into the lack of gmp/mpfr/mpc pretty
quickly, so maybe those ought to be a priority...

...

Wow!  It looks like I already solved the gmp problem!

...

Coming right along.  Buttons, soft buttons, and keyboard buttons are all
good now, and most of the C code compiles in the JNI context now.  The
.so is large now, so it does seem to successfully be pulling in gmp/etc.


November 28, 2011.

A minor innovation in UI design.  Now that the root menu buttons are not
always displayed, I can abandon the arbitrary limit of 4 of them.  In
particular, I can get rid of :misc and drag :misc:edit and :misc:units
into the top-level.

Also, let's prioritize getting an 'undo' in.


December 10, 2011.

Okay, I've implemented most of the STO/REF sort of functionality.
However, I'm dismayed by how many general things I have to soak up here
in the menu thing.  I'm really looking at this 'alternate function' sort
of stuff I'm throwing into the menu, to see if there is maybe a better
one.

Question 1: Display alternate functions?
  (a) at all times
  (b) only when alternate mode is selected
Question 2: How to activate?
  (a) hard-button: option
  (b) hard-button: search
  (c) dedicated soft-button on screen
  (d) long press

The status quo is 1(b) 2(a), using the default android 6-element menu
with awkward "more" option.  I'm actually alright with that answer, it
just can't be the default awkward menu.

But since I think this has a real potential of overlooking things, and/or
of ballooning, let's attempt to make a complete list:

  variable management: 7
     store (val name -- )
     mkdir (name -- )
     store (val click -- )
     reference (click -- name)
     value (click -- val)
     edit (click -- editor)
     erase (click -- )
  starting new types: 4
     "     (string)
     #     (literal)
     []    (array)
     <<>>  (function)
  misc: 2
    undo
    edit top of stack
  
Oh and keep in mind that the reset soft84rc function has to go in there
somewhere...

A 4x4 grid would suffice for that, and would have 3 spots left-over. Of
course, coming up with brilliant SHORT names and/or icons for these would
be fantastic.

It seems like kind of a clusterfuck, but the HP48 wasn't that much
better.  Basically, you would hit one shift key (shift-up) and it would
"store (val click --)" and another (shift-down) and it would "value
(click -- val)".  And if you started string mode before hitting the
thing, then it would do "reference (click -- name)".  Since my references
aren't necessarily strings, I'm not sure I like the latter hack.

...

It got a little easier, I declared by fiat that I would not bother to add
an edit mode for a variable...you would put it on the stack, edit it, and
then store it manually.
  
...

That seems to work, so I'm on to making it pop up the android soft
keyboard...  Easy!  Yay.


December 11, 2011.

Okay, I fine-tuned the 'alternate function' buttons and I'm pretty happy
with how they function for the most part now.

So I am down to "implement undo" on my todo list, which is not an
inspirational place to be so I haven't done it.  Though I have discovered
that the HP48 only offers one level of undo, so that makes me more
confident.

However, undo kind of raises the spectre of reference counting.  And I
have independently discovered another scenario that calls out for
reference counting.

Suppose you make a directory :user:a and within that, a variable
:user:a:b.  Then you put a reference to the variable on the stack (a
vk_call).  Then you delete the variable and the directory.  Deleting the
directory causes the directory itself to only become a vark_undefined,
but the members of the directory are free()d (free_variable_list()).

I'm not sure why I did that, really.  But anyways, the effect is that the
vk_call points to a variable that has now been free()d and will
eventually cause a SIGSEGV as the memory manager progresses.

At first, this looks like an excellent opportunity for reference
counting.  But honestly, I think it just means that we can't actually
free() a variable record, we would just set it to vark_undefined.  At
first, this seems dangerous because it is going to leak memory if you
create and delete a single variable over and over again.

But maybe with some hacking it can become a good idea.  The saving grace
is that a vark_undefined stays in the dictionary, so if it is erased and
then re-created then it will simply re-use the same variable_rec.

The problem is that if you delete the directory, there is nowhere in the
dictionary for the variable to remain.  I could make a special
vark_undefined_directory but if it is re-defined as a non-directory then
we're stuck at the same spot.

So I can decide to leak variable_recs when we erase a directory, which is
certainly good enough for today.  Or we can add reference
counting...might even add reference counting to value_recs and then the
stack undo feature gets majorly-decreased overhead.

I'm trying to imagine the sort of code that will make directories in a
loop, and it's seeming like even if it wastes one variable_rec (12 bytes)
each time through, it'd have to go a million times just to waste 12MB,
which is truly not that large a quantity of memory even on a cellphone.
And I can't really imagine anyone doing a million loops, if they do there
will be many performance limiters outside of this memory leak.

On the other hand, reference counting may be very simple and clean with
almost no overhead and, most importantly, little code change.  And it
would really be slick if the values, at least, were ref-counted...  A
classic waffle!

...

Okay, I sold myself on it, so I added the reference counts.  I am still
convinced it was the right thing to do but I'm going to highlight two
problems that might remain.  First, it doesn't help us with "undo"
because, despite reference counting, everything will still side-effect
into these values.  Second, it doesn't help us with vk_calls sitting
around too much if the containing directory is deleted.  That's because
the name of the variable depends upon the containing directory.  So if
you delete "user:a" then "user:a:b" becomes just "b" with no location
within the directory hierarchy.  Anything that still references it
directly can save and load it, but it won't be visible to anything new.

But really all that means is that if the user intends to delete a
directory and still reference variables inside of it, he will have to use
string names rather than direct references.  Which is kind of a bridge
we've already crossed so.. *shrug* I'm happy with it.

Anyways, the reference counting is definitely necessary.  There are links
from the stack/value side of things, and from the dictionary....in order
to handle that, we really need some minimal reference counting.

...  And it turns out undo is super easy, just using the value_list
primitives.  The hardest question was where to call it.  I declared that
I would call it for any press_variable() of a vk_value or vk_builtin, but
for keyhandlers I would explicitly call it within the keyhandler if it
happened to touch the stack.


January 2, 2011.

I spent most of my vacation in Arizona and didn't feel I had enough time
to get much done on this before going back to work.  Nonetheless here I
am the night before work begins anew.

Neil pointed out that the back button in my application violates his
POLA.  I decided that I never use it and it's redundant to "UP" as well,
so let's just toss it.  Default back button.


XXX - adding 0_degK doesn't convert to kelvins, it converts to celsius
XXX - need a way to edit within a member in a list or program...maybe :keys:edit and :keys:st_list/st_func should work in edit mode??

XXX - make some documentation

XXX - why does multiplying "acos(err)" by 2 cause prog:dict:acos(err) (a call) to appear on the stack??

XXX - settable precision, and display precision
XXX - programming control flow, scoping (local variables)
XXX - plotting
XXX - why does mpc cos/cosh lock up if you repeat them?  seems to be related to a very small number (epsilon) times i.
XXX - flush out math menus: calculus, vector, matrix, list, hyp(expm,lnp1), real, base, plot, prob, fft
XXX - flush out prog menus: control, dictionary, compare, type, file, list, io
XXX - modal features? (base, precision, ??degrees??)
XXX - cas
gglx - make the val list highlighting code just change the affected areas of the screen instead of all of it
gglx - still overflow multidisp buffer sometimes (too wide)
gglx - limit width of string in softbutton, such as prog:dict:e:edit_stack
gglx - less shitty font, especially for + - * /
gglx - do something with stack highlighting (fancy pick?), or drop it

XXX - things that should be easier: misc:units
