i am also having this issue but only for one user account while for the other user account on the same machine it just works.
when looking into console it can be seen that there is a crash in SiriPreferenceExtension
Process: SiriPreferenceExtension [1530]
Path: /System/Library/ExtensionKit/Extensions/SiriPreferenceExtension.appex/Contents/MacOS/SiriPreferenceExtension
Identifier: com.apple.Siri-Settings.extension
Version: 1.0 (1)
Build Info: SiriPlugins-3302005001000000~14
Code Type: ARM-64 (Native)
Parent Process: launchd [1]
Date/Time: 2024-01-15 20:10:09.0732 +0100
OS Version: macOS 14.2.1 (23C71)
Crashed Thread: 0 Dispatch queue: com.apple.main-thread
Exception Type: EXC_CRASH (SIGABRT)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Termination Reason: Namespace SIGNAL, Code 6 Abort trap: 6
Terminating Process: SiriPreferenceExtension [1530]
Application Specific Information:
abort() called
Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0 libsystem_kernel.dylib 0x183ace0dc __pthread_kill + 8
1 libsystem_pthread.dylib 0x183b05cc0 pthread_kill + 288
2 libsystem_c.dylib 0x183a11a40 abort + 180
3 libswiftCore.dylib 0x1937416c8 swift::fatalErrorv(unsigned int, char const*, char*) + 128
4 libswiftCore.dylib 0x1937416e8 swift::fatalError(unsigned int, char const*, ...) + 32
5 libswiftCore.dylib 0x1937353d4 swift::swift_dynamicCastFailure(void const*, char const*, void const*, char const*, char const*) + 80
6 libswiftCore.dylib 0x193735450 swift::swift_dynamicCastFailure(swift::TargetMetadata<swift::InProcess> const*, swift::TargetMetadata<swift::InProcess> const*, char const*) + 124
7 libswiftCore.dylib 0x19373aad8 swift_dynamicCast + 344
8 SiriPreferenceExtension 0x1048c1360 0x104898000 + 168800
9 SiriPreferenceExtension 0x1048c0f28 0x104898000 + 167720
10 SiriPreferenceExtension 0x1048c00ac 0x104898000 + 164012
11 libdispatch.dylib 0x183954910 _dispatch_client_callout + 20
12 libdispatch.dylib 0x18395614c _dispatch_once_callout + 32
13 SiriPreferenceExtension 0x1048c0140 0x104898000 + 164160
14 SiriPreferenceExtension 0x1048c6100 0x104898000 + 188672
15 SwiftUI 0x1af4dfa3c 0x1ae761000 + 14150204
16 SwiftUI 0x1aef6106c 0x1ae761000 + 8388716
17 SwiftUI 0x1aef60778 0x1ae761000 + 8386424
18 SwiftUI 0x1afff2758 0x1ae761000 + 25761624
19 SwiftUI 0x1afff2798 0x1ae761000 + 25761688
20 libswiftCore.dylib 0x193400230 withUnsafeMutablePointer<A, B>(to:_:) + 28
21 SwiftUI 0x1afff25ec 0x1ae761000 + 25761260
22 SwiftUI 0x1afff2214 0x1ae761000 + 25760276
23 SwiftUI 0x1afa74b6c 0x1ae761000 + 20003692
24 AttributeGraph 0x1b05ab7a4 AG::Graph::UpdateStack::update() + 512
25 AttributeGraph 0x1b05abfe0 AG::Graph::update_attribute(AG::data::ptr<AG::Node>, unsigned int) + 424
26 AttributeGraph 0x1b05b49f8 AG::Graph::input_value_ref_slow(AG::data::ptr<AG::Node>, AG::AttributeID, unsigned int, unsigned int, AGSwiftMetadata const*, unsigned char&, long) + 720
27 AttributeGraph 0x1b05ccc40 AGGraphGetValue + 228
28 SwiftUI 0x1afff15d8 0x1ae761000 + 25757144
29 SwiftUI 0x1afff1be0 0x1ae761000 + 25758688
30 SwiftUI 0x1afff17c4 0x1ae761000 + 25757636
....
140 AppKit 0x1873d56f4 +[NSAnimationContext runAnimationGroup:] + 56
141 SwiftUI 0x1af609960 0x1ae761000 + 15370592
142 SwiftUI 0x1af609f58 0x1ae761000 + 15372120
143 AppKit 0x187e50af4 ___NSViewLayout_block_invoke + 592
144 AppKit 0x187400714 NSPerformVisuallyAtomicChange + 108
145 AppKit 0x187405244 _NSViewLayout + 96
146 AppKit 0x187e470cc __36-[NSView _layoutSubtreeWithOldSize:]_block_invoke + 364
147 AppKit 0x187400714 NSPerformVisuallyAtomicChange + 108
...