Ragwort is a generally seen as a weed. It is toxic to horses. To get it out dig out the roots; some of these weeds have very long root structures so dig deep down as if you are getting rid of a dandelion.
Ragwort does provide a home for various caterpillars though so you might want to keep it.
Foxgloves contain digitalis which is poisonous in quite small amounts so you may want us to dig it up and get rid of your foxgloves. Foxgloves are excellent for bees and quite attractive to some so we recommend keeping some for this. Foxgloves love shade so they provide something for areas of the garden where not much seems to go.
You can buy special varieties of foxglove from the garden centre or just rely on the wild ones that naturally pop up in your garden.
If you don't want foxgloves they are easy to pull out.
Planting this as a wildflower has become popular in recent years. When allowed to grow teasel create very interesting plants so naturalists should plant these.
Teasel look ugly when undeveloped so we are asked to remove these plants most of the time but let them grow and you would get a surprise. Butterflies and bees love these and the cones look nice in the snow in winter. This plant is a weed in formal gardens but it definitely has a place in informal gardens.