Apple Intelligence is now available on iPhone, iPad, and Mac!

Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

problem with import of XML Apple HealthKit Export Version: 12

For several years I have been exporting my Apple Health Export to XML and then importing it into R for analysis. I just upgraded to iOS 16.0. I'm using the same code I used in August, but now I get the following error:


read_xml.character("~/Downloads/apple_health_export/export.xml") :  
ATTLIST: no name for Attribute [68]


Given that I just did the upgrade I thought I'd see if anyone else was having a new problem with the export. I've tried a couple of different R packages to do the import with no luck.


It's HealthKit Export Version: 12


My export.xml is now 1.7 GB so it makes it awkward to try to check it out with a text editor.


Any suggestions?


I've done quite a bit of work with the XML export previously. Here are some blog posts on that topic. I'm hoping I'm not alone having a new problem with this version. I just did the update this afternoon, so I suppose my next step is to try the most sophisticated technique I know of and turn my phone off and turn it back on and hope for something different.

iPhone 13 Pro

Posted on Sep 16, 2022 4:56 PM

Reply
Question marked as Top-ranking reply

Posted on Sep 18, 2022 1:45 PM

OK. I finally upgraded to iOS 16. I saved a copy of my iOS 15 Health Data for comparison.


I usually try to give Apple the benefit of the doubt. But this case, Apple completely screwed this up. Not even close.


First of all, Apple is still using DTD files. That's surprising. Plus, it is embedded rather than referencing a URL. That's unusual. Most importantly, Apple scrambled the syntax so it won't ever work. Note to Apple, use a schema instead and do it correctly. No other way to say it.


Edit: in addition to the above, Apple flat-out did the DTD incorrectly. I fixed the scrambled syntax and was able to get "xmllint" happy, but it still failed validation. So I went ahead and fixed the DTD too.


Next, Apple scrambled the output of some elements, repeating the "startDate" attribute twice. The standard rule of thumb is to use a proper XML-processing library to generate XML for output. But those old XML libraries are cryptic and difficult to use. And sometimes, vendors who provide XML-processing libraries release buggy ones (that dig's for you, too, Apple 😄 ). So often, developers take shortcuts and just spit out text that they hope will be valid XML. Full disclosure, I do this myself. But if you roll your own, you've got to do it correctly. Apple failed at that.


There are two distinct fixes you will have to apply. I will write this out as a longer sequence of steps just to make things crystal clear.

1) Export your health data. I saved mine to my iCloud Drive. You'll definitely need a Mac to fix this.

2) Go into your iCloud Drive, or wherever you saved the file and expand the "export.zip" file. It will be expanded into a folder named "apple_health_export". For ease of description, I copied my file to "/tmp" and exported it there.

3) In the Terminal, navigate to that folder using "cd /tmp/apple_health_export". (Not including quotes)

4) Create a new filed called "patch.txt" with the following content:

--- export.xml	2022-09-18 15:17:09.000000000 -0400
+++ export-fixed.xml	2022-09-18 16:37:08.000000000 -0400
@@ -15,6 +15,7 @@
   HKCharacteristicTypeIdentifierBiologicalSex       CDATA #REQUIRED
   HKCharacteristicTypeIdentifierBloodType           CDATA #REQUIRED
   HKCharacteristicTypeIdentifierFitzpatrickSkinType CDATA #REQUIRED
+  HKCharacteristicTypeIdentifierCardioFitnessMedicationsUse CDATA #IMPLIED
 >
 <!ELEMENT Record ((MetadataEntry|HeartRateVariabilityMetadataList)*)>
 <!ATTLIST Record
@@ -39,7 +40,7 @@
   startDate     CDATA #REQUIRED
   endDate       CDATA #REQUIRED
 >
-<!ELEMENT Workout ((MetadataEntry|WorkoutEvent|WorkoutRoute)*)>
+<!ELEMENT Workout ((MetadataEntry|WorkoutEvent|WorkoutRoute|WorkoutStatistics)*)>
 <!ATTLIST Workout
   workoutActivityType   CDATA #REQUIRED
   duration              CDATA #IMPLIED
@@ -63,7 +64,7 @@
   duration             CDATA #IMPLIED
   durationUnit         CDATA #IMPLIED
 >
-<!ELEMENT WorkoutEvent EMPTY>
+<!ELEMENT WorkoutEvent (MetadataEntry?)>
 <!ATTLIST WorkoutEvent
   type                 CDATA #REQUIRED
   date                 CDATA #REQUIRED
@@ -79,6 +80,7 @@
   minimum              CDATA #IMPLIED
   maximum              CDATA #IMPLIED
   sum                  CDATA #IMPLIED
+  unit                 CDATA #IMPLIED
 >
 <!ELEMENT WorkoutRoute ((MetadataEntry|FileReference)*)>
 <!ATTLIST WorkoutRoute
@@ -153,6 +155,7 @@
   dateIssued       CDATA #REQUIRED
   expirationDate   CDATA #REQUIRED
   brand            CDATA #IMPLIED
+>
 <!ELEMENT RightEye EMPTY>
 <!ATTLIST RightEye
   sphere           CDATA #IMPLIED
@@ -203,13 +206,6 @@
   diameter         CDATA #IMPLIED
   diameterUnit     CDATA #IMPLIED
 >
-  device           CDATA #IMPLIED
-<!ELEMENT MetadataEntry EMPTY>
-<!ATTLIST MetadataEntry
-  key              CDATA #IMPLIED
-  value            CDATA #IMPLIED
->
->
 ]>
 <HealthData>
  <ExportDate/>

Ignore the colours. That is just something the forum software adds to a code block.

5) Run the following command to patch your XML export file: "patch < patch.txt". (Again, omit the quotes.) The output should look like this:

patching file export.xml
Hunk #6 succeeded at 206 with fuzz 2.

Don't worry about that hunk #6 warning. I had to manually hack the patch file since it originally included two lines of content, which would be different. Luckily, patch gracefully handles it.

All this command does is replace the DTD data with something correct.


Similar questions

35 replies
Question marked as Top-ranking reply

Sep 18, 2022 1:45 PM in response to guilford1639

OK. I finally upgraded to iOS 16. I saved a copy of my iOS 15 Health Data for comparison.


I usually try to give Apple the benefit of the doubt. But this case, Apple completely screwed this up. Not even close.


First of all, Apple is still using DTD files. That's surprising. Plus, it is embedded rather than referencing a URL. That's unusual. Most importantly, Apple scrambled the syntax so it won't ever work. Note to Apple, use a schema instead and do it correctly. No other way to say it.


Edit: in addition to the above, Apple flat-out did the DTD incorrectly. I fixed the scrambled syntax and was able to get "xmllint" happy, but it still failed validation. So I went ahead and fixed the DTD too.


Next, Apple scrambled the output of some elements, repeating the "startDate" attribute twice. The standard rule of thumb is to use a proper XML-processing library to generate XML for output. But those old XML libraries are cryptic and difficult to use. And sometimes, vendors who provide XML-processing libraries release buggy ones (that dig's for you, too, Apple 😄 ). So often, developers take shortcuts and just spit out text that they hope will be valid XML. Full disclosure, I do this myself. But if you roll your own, you've got to do it correctly. Apple failed at that.


There are two distinct fixes you will have to apply. I will write this out as a longer sequence of steps just to make things crystal clear.

1) Export your health data. I saved mine to my iCloud Drive. You'll definitely need a Mac to fix this.

2) Go into your iCloud Drive, or wherever you saved the file and expand the "export.zip" file. It will be expanded into a folder named "apple_health_export". For ease of description, I copied my file to "/tmp" and exported it there.

3) In the Terminal, navigate to that folder using "cd /tmp/apple_health_export". (Not including quotes)

4) Create a new filed called "patch.txt" with the following content:

--- export.xml	2022-09-18 15:17:09.000000000 -0400
+++ export-fixed.xml	2022-09-18 16:37:08.000000000 -0400
@@ -15,6 +15,7 @@
   HKCharacteristicTypeIdentifierBiologicalSex       CDATA #REQUIRED
   HKCharacteristicTypeIdentifierBloodType           CDATA #REQUIRED
   HKCharacteristicTypeIdentifierFitzpatrickSkinType CDATA #REQUIRED
+  HKCharacteristicTypeIdentifierCardioFitnessMedicationsUse CDATA #IMPLIED
 >
 <!ELEMENT Record ((MetadataEntry|HeartRateVariabilityMetadataList)*)>
 <!ATTLIST Record
@@ -39,7 +40,7 @@
   startDate     CDATA #REQUIRED
   endDate       CDATA #REQUIRED
 >
-<!ELEMENT Workout ((MetadataEntry|WorkoutEvent|WorkoutRoute)*)>
+<!ELEMENT Workout ((MetadataEntry|WorkoutEvent|WorkoutRoute|WorkoutStatistics)*)>
 <!ATTLIST Workout
   workoutActivityType   CDATA #REQUIRED
   duration              CDATA #IMPLIED
@@ -63,7 +64,7 @@
   duration             CDATA #IMPLIED
   durationUnit         CDATA #IMPLIED
 >
-<!ELEMENT WorkoutEvent EMPTY>
+<!ELEMENT WorkoutEvent (MetadataEntry?)>
 <!ATTLIST WorkoutEvent
   type                 CDATA #REQUIRED
   date                 CDATA #REQUIRED
@@ -79,6 +80,7 @@
   minimum              CDATA #IMPLIED
   maximum              CDATA #IMPLIED
   sum                  CDATA #IMPLIED
+  unit                 CDATA #IMPLIED
 >
 <!ELEMENT WorkoutRoute ((MetadataEntry|FileReference)*)>
 <!ATTLIST WorkoutRoute
@@ -153,6 +155,7 @@
   dateIssued       CDATA #REQUIRED
   expirationDate   CDATA #REQUIRED
   brand            CDATA #IMPLIED
+>
 <!ELEMENT RightEye EMPTY>
 <!ATTLIST RightEye
   sphere           CDATA #IMPLIED
@@ -203,13 +206,6 @@
   diameter         CDATA #IMPLIED
   diameterUnit     CDATA #IMPLIED
 >
-  device           CDATA #IMPLIED
-<!ELEMENT MetadataEntry EMPTY>
-<!ATTLIST MetadataEntry
-  key              CDATA #IMPLIED
-  value            CDATA #IMPLIED
->
->
 ]>
 <HealthData>
  <ExportDate/>

Ignore the colours. That is just something the forum software adds to a code block.

5) Run the following command to patch your XML export file: "patch < patch.txt". (Again, omit the quotes.) The output should look like this:

patching file export.xml
Hunk #6 succeeded at 206 with fuzz 2.

Don't worry about that hunk #6 warning. I had to manually hack the patch file since it originally included two lines of content, which would be different. Luckily, patch gracefully handles it.

All this command does is replace the DTD data with something correct.


Sep 18, 2022 1:46 PM in response to etresoft

No, you're not done yet! I just hit the 5000 character posting limit. Here are the rest of the instructions:


6) Now, fix those duplicate dates. I normally avoid tools like this, but it's quick and easy. Run the following command in the Terminal: "sed 's/startDate/endDate/2' export.xml > export-fixed.xml". As before, omit the double quotes. Just be careful with the inner quotes and make sure they don't get turned into smart quotes. This command processes every line and fixes the second occurrence of "startDate" with "endDate".


I tested this with the xmllint tool and it is happier now. It even validates if you include then "--valid" flag, like so: "xmllint --noout --valid export-fixed.xml". If you get no output, the XML is valid.


You should be able to import from the "export-fixed.xml" file at this point.


I don't know what the rest of your procedure is, so I don't know if you will encounter additional problems. There may be another exported XML file "export_cda.xml" that depends on a complicated hierarchy of schema documents, which are not included. It looks like it would be a royal pain to track those down.

Oct 17, 2022 8:30 AM in response to urbanspark

urbanspark wrote:

Unfortunately I'm not a programmer so I don't know what I can do and what I can't do... I guess I'll have to wait until Apple will fix this problem :/


You can either learn about and fix the XML, or can log a bug report and wait for an update from Apple or from the app vendor that fixes this.


To log a report with Apple, include the URL to this thread, and also with your particular apps and XML errors, here: Product Feedback - Apple

.

Oct 30, 2022 9:07 AM in response to bmk1984

bmk1984 wrote:

I'm new to this environment, can you tell me where this sort of thing should be logged (where they will notice)?


If you’re enrolled in the Apple Developer program, log a problem report that way:

Bug Reporting - Apple Developer


If you’re not enrolled in the Developer program, log feedback this way:

Product Feedback - Apple


Provide the xmllint command and output, and/or details of the specific XML formatting errors encountered, and iOS version.


Oct 18, 2022 5:12 PM in response to guilford1639

I tried using export.xml in python and encountered a few errors:

  • First two were missing or out-of-place characters in the data definition part part. These were fixable with simple edits.
  • Third was a duplicate tag: startDate/endDate really has two startDate tags.

...ActiveEnergyBurned" startDate="2020-12-28 16:34:27 -0400" startDate="2020-12-28 16:54:59 -0400"...

There are too many to fix manually, so I'll wait for an Apple update..


You've gotta wonder how this sort of thing got by the QA crew? I'm analyzing the ECG data (which is in separate "funny CSV" files), and the errors mentioned suggests that no actual testing was done?


Sep 17, 2022 11:31 AM in response to guilford1639

Following comment from etresoft, I tried xmllint and got the following error below. Given this happened right after a major iOS upgrade I'm going to assume there's an actual problem with the export's XML format. I'll give it some time before I go mucking around in 1.5 gigabytes of XML data trying to see if I can fix it well enough that it will load. I'm not aware of there being any eye related data in my health record. Maybe I'll come back to this later to try to figure out what else I can do with xmllint to explore the format of the data. In the meantime I'll assume the problem is at the Apple end of things and wait for the next update and try again.


xmllint /Users/johngoldin/Downloads/apple_health_export/export.xml
/Users/johngoldin/Downloads/apple_health_export/export.xml:156: parser error : ATTLIST: no name for Attribute
<!ELEMENT RightEye EMPTY>

Sep 17, 2022 12:22 PM in response to guilford1639

That is used for contact lens information.


Since the XML is invalid, you probably won't be able to use any XML tools on it. You could use grep, with something like this:


grep RightEye /Users/johngoldin/Downloads/apple_health_export/export.xml


And that should spit out a small section of the file. The error is most likely on the same line. If you paste that into a reply, we could tell you how to fix it.

Sep 17, 2022 12:32 PM in response to etresoft

Or grep -v {target} the file (-v to invert the match) and ditch the bad line, if it’s all on the same line.


Otherwise, you’re going to be temporarily patching the XML with some scripting, or tweak the DTD.


I don’t see an easy way to drop the problem syntax with xmllint, but there’s probably another XML tool that can. Or libxml2 or,ilk can read and process it from within an app.

Oct 16, 2022 11:42 PM in response to etresoft

Hey @all, I think I have the same problem. Everytime when I want to open the export file I get these error messages:


This page contains the following errors:

error on line 156 at column 1: ATTLIST: no name for Attribute

error on line 206 at column 3: internal error: xmlParseInternalSubset: error detected in Markup declaration


Below is a rendering of the page up to the first error.



I don't have a Mac, only a windows pc, so is there any fix for me too?

Oct 17, 2022 5:01 AM in response to urbanspark

You seem to have forgotten to include your attachment.


All of the fixes I described can be done on Windows. Windows has an excellent array of XML tools. Fixing the duplicate attribute will require some kind of advanced text editor, but you should have those too. Unfortunately, I’m not familiar with that platform so I can’t give you specifics.

Oct 17, 2022 6:33 AM in response to etresoft

There is no attachment, I just get this messages in a red box:


"This page contains the following errors:

error on line 156 at column 1: ATTLIST: no name for Attribute

error on line 206 at column 3: internal error: xmlParseInternalSubset: error detected in Markup declaration


Below is a rendering of the page up to the first error."


There is nothing more on the page. Just this red box with the text.


Unfortunately I'm not a programmer so I don't know what I can do and what I can't do... I guess I'll have to wait until Apple will fix this problem :/

Oct 29, 2022 2:43 AM in response to etresoft

Thanks etresoft!


For my export today on iOS 16.1 it was enough to apply your patch. There weren't any duplicate `startDate`s and the patch didn't throw any warnings either. The resulting XML is still not valid but I could process it the same way I did on iOS 15. Maybe Apple changed fixed a few things in iOS 16.1?


problem with import of XML Apple HealthKit Export Version: 12

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple Account.