I see what you mean.
In such a case, I think you might be better off creating a custom widget using the action pad itself and then positioning it to the top-right of the map view
view.ui.add(YOUR_CUSTOM_WIDGET_REFERENCE_HERE, {
position: "top-right"
});
This should get you what you need I think.