What is the exact message that you see?
How are you trying to import the file? This is what I see when I check iBooks Author help
Add a 3D widget
Choose 3D from the Widgets pop-up menu in the toolbar.
To add a file, do one of the following:
Drag a file from the Finder to the widget on the page.
In the Widget inspector, click Interaction, click Choose, and choose a COLLADA file with the extension .dae.
To constrain the object’s rotation to horizontal or vertical axes, choose an option from the Object Rotation pop-up menu.
To automatically rotate the object, select “Auto-rotate object when idle.”