Gijs,
So in your code for your toolbar at the bottom of the widget you need to have a css rule with higher specificity that will use !important to override the
.jimu-widget-frame.jimu-container *
So hopefully your toolbar as a class applied to it and you can just add a rule to it
This is not just copy and paste code, it is just an example of a higher specificity css rule.
.jimu-widget-frame.jimu-container .mytoolbar.button {
color: green !important;
}