Apple Event: May 7th at 7 am PT

Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

AppleScript Numbers: design options for performance

Environment:

  • Mojave , Numbers 6.1, Automator 2.9
  • Both Numbers and Automator are set via System Preferences > Security & Privacy > Accessibility


Background:

I have a spreadsheet for finances in which one sheet ("Monthly Summary") has ~40 rows of data categorization (e.g.: medical expenses, groceries, transportation, etc.) and N columns of months for 4.5 years (currently, e.g.: 54 columns and growing).


The data is pulled from another sheet ("Details") which has many rows (over 4000) with 3 columns:

    • Date
    • Amount
    • Categorization Key


My current Service script design uses =SUMIFS(...) formulas in each cell of the "Monthly Summary" table to extract and sum-up the relevant data for that cell based on the row's categorization key and the columns year/month date specification.


Issue:

Numbers sometimes stalls for quite a while "calculating".

I'm wondering if alternate design might be better.


Option 1: Current Design

As indicated, right now I have this massive sheet with over 2000 =SUMIFS(...) inserted into it.

      • Pros: Already set up as such; Code already written
      • Cons: Periodic usability delays when Numbers decides to start calculations - press,ably for each formula encountered in the spreadsheet.

Option 2: Separate Tables Per Year

Someone suggested I have multiple tables on a sheet, one for each year so that any single table would have about 500 =SUMIFS(...) in it.

      • Pros: Not sure, perhaps by having less formulas per table the calculations take less time?
      • Cons: Personally I find it confusing to navigate multiple tables on a single sheet

Option 3: Separate Sheets Per Year

This is essentially the same as Option 2 except that I have separate sheets with one table each, for each year, but still about 500 =SUMIFS(...) on each sheet / table.

      • Pros: Not sure, perhaps by having less formulas per sheet/table the calculations take less time?
      • Cons: Not sure. A bit harder to visually scan the "big picture" because you have to flip back-and-forth between sheets.

Option 4: Copy Formula Results Over Formula

From my own trials I discoed that once I set the formula in the cell, it gets calculated, and I can then copy the value of the cell over itself - e.g.:

 ...
 set value of targetCell to regexString
 set value of targetCell to value of targetCell
 set the format of targetCell to currency
 ...
      • Pros: Complete elimination of formulas in the spreadsheet; can keep all data on one sheet or break it up into multiple sheets / tables as seems reasonable.
      • Cons: The "Monthly Summary" data would not be automatically kept up-to-date as more data is entered into the "Details" sheet, so the Service script would need to be re-run periodically.

Option 5: Perform Calculations In Script Instead of Formulas

I haven't looked into this too closely yet, mostly because I'm not that comfortable doing full-out programming in AppleScript (I have 20+ years experience writing Perl code and I often find it difficult to change my coding concepts to AppleScript structures - if possible) - but essentially I would perform all the same functions that the =SUMIFS(...) formula is doing in "pure code" and then insert the result into each cell .

      • Pros: Complete elimination of formulas in the spreadsheet; can keep all data on one sheet or break it up into multiple sheets / tables as seems reasonable.
      • Cons: The "Monthly Summary" data would not be automatically kept up-to-date as more data is entered into the "Details" sheet, so the Service script would need to be re-run periodically; It is also conceivable that the amount of time it takes to perform the equivalent of the formula internally may equal, or even exceed the amount of time that Numbers now spends "calculating" the results of the existing formulas.


At this point I'm interested in opinions - pros & cons, etc.


I'm going to move ahead with Option 1 because I have it pretty much ready to go, and Option 4 is a trivial two-line addition to the script, but I'm curious to hear what other's think to see if I'm overlooking something.

Apple Watch Series 4, watchOS 5

Posted on Jul 13, 2019 7:38 PM

Reply
Question marked as Best reply

Posted on Jul 14, 2019 1:18 AM


Question: How much of your original data from 2005 is going to change this year? What calculations relevant to this year do you need to (RE)do this year?


I'm assuming the answer to both questions is "None." If that assumption is correct, the next question is 'For what years other that 2005 does that question have the same answer?'


Apply Option 4 to all of those years, either by using a script (which you'll need to run only once on each table being 'retired' to fixed values status) or by selecting ALL cells on a table to be 'retired', Copying, then, with all cells still selected, going Edit (menu) > Paste Formula Results, to replace all formulas on the table with the last result calculated by that formula (and with non-formula cells, pasting the same value in as was just copied from that cell.


Eliminating all the now unnecessary (re)calculation of known results should reduce the delaays.


Regards,

Barry



Similar questions

5 replies
Question marked as Best reply

Jul 14, 2019 1:18 AM in response to astoller


Question: How much of your original data from 2005 is going to change this year? What calculations relevant to this year do you need to (RE)do this year?


I'm assuming the answer to both questions is "None." If that assumption is correct, the next question is 'For what years other that 2005 does that question have the same answer?'


Apply Option 4 to all of those years, either by using a script (which you'll need to run only once on each table being 'retired' to fixed values status) or by selecting ALL cells on a table to be 'retired', Copying, then, with all cells still selected, going Edit (menu) > Paste Formula Results, to replace all formulas on the table with the last result calculated by that formula (and with non-formula cells, pasting the same value in as was just copied from that cell.


Eliminating all the now unnecessary (re)calculation of known results should reduce the delaays.


Regards,

Barry



Jul 14, 2019 7:04 AM in response to astoller

Option 5 would likely involve a lot of Apple Events, which would be slow.


It does appear that Option 4 would be the way do go. Bear in mind that Numbers, unlike Excel, is not designed to handle large datasets, and "wide" sheets tend to be unwieldy. So as you add months you'll be pushing the limits.


If possible you might be worth considering "archiving" the oldest data in a separate document.


SG

Jul 14, 2019 3:17 PM in response to SGIII

If I find it starts to get bogged-down scrolling horizontally, I'll either move old year's of summary information to distinct sheets/tables or just remove them (this is for my own purposes so there's no "law" or "regulation" that needs to be adhered to for record-keeping, etc.)


I'm going to leave this thread "open" for a little while in case anyone else wants to add their $0.02, then I'll mark something as "Solved"

Jul 14, 2019 3:21 PM in response to astoller

astoller wrote:

If I find it starts to get bogged-down scrolling horizontally, I'll either move old year's of summary information to distinct sheets/tables or just remove them (this is for my own purposes


Moving data to distinct sheets/tables in the same document may help a little. But if you've got a lot of data, even in separate sheets/tables, performance in Numbers could suffer. Moving the table/sheets to a separate "archive" document (or simply deleting them) should definitely help.



SG

AppleScript Numbers: design options for performance

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple ID.