How to set NOT to "move object with text" as default?

Although you can't change the default for "Move object with text" in the dialog, you can run this macro that changes the setting for every shape (floating graphic) in the document.

Sub DontMoveWithText()
    Dim shp As Shape
    For Each shp In ActiveDocument.Shapes
        shp.RelativeVerticalPosition = wdRelativeVerticalPositionPage
    Next shp
End Sub

If you install it in your Normal.dotm template or in an add-in (a template stored in the %appdata%\Microsoft\Word\Startup folder) and assign a keyboard shortcut or a Quick Access Toolbar button to it, you can run it at any time after adding images. It also won't hurt anything to run it multiple times on the same document.

Since Word considers text boxes (from Insert > Text Box) to be a kind of Shape object, the macro above will also affect them. If you want to exclude text boxes from this treatment, use this code instead:

Sub DontMoveWithText()
    Dim shp As Shape
    For Each shp In ActiveDocument.Shapes
      If Not shp.Type = msoTextBox Then
        shp.RelativeVerticalPosition = wdRelativeVerticalPositionPage
      End If
    Next shp
End Sub