If we haven't inited the mm code, don't try and tear it down. Signed-off-by: Dave Airlie <airlied@redhat.com>