Do you have matched pairs of DIMM cards installed?
Accessing RAM has 2 phases. The first is draining the memroy cell to see if it had anything in it, and the second is refilling the memory cell now that it has been drained.
If RAM DIMMs are installed as matched pairs (where matched means same size and technical specs), then the hardware will arrange for the RAM to be accessed first from one DIMM, then while it is restoring the drained values, it will access the paired DIMM. This alternating between DIMMs allows for more data to be transferred from RAM to the CPU at a faster rate.
If the DIMMs are not matched, the hardware will access RAM serially, waiting between each read for the previous read to refill the drained memory cells.
Matched pairs does not double speed, as the CPU is not always accessing RAM, as the CPU has its own on-chip cache to defer going to RAM, but it does improve performance.
If you did not properly pair your DIMMs when you installed them, that might explain some of your issues.
Or I could be totally wrong, and you are experiencing some other problem.