I think I just found a solution from the archives of this forum.
Goto Firefox Preferences > Advanced > General and uncheck 'Use hardware acceleration when available'.
Now when I have Firefox on my DisplayLink monitor the DisplayLinkManager process consumes less than 5% CPU.
reference:
http://www.displaylink.org/forum/showthread.php?t=730