Descartes argues that God could not be a deciever and so his clear and distinct ideas, which presented themselves to him as self-evidently true, really were necessary truths. If it was the case that Descartes had this strong belief that there are physical objects when there weren’t any really then God would be the Evil Demon; but that isn’t possible. God wouldn’t allow Descartes to be decieved in this way. I often joke that Descartes must not have read the Book of Job because God does allow Job to be decieved (though, it is true that God is not the one doing the decieving) into thinking that it is God who is the one responsible for Job’s misfortunes. But actually, after having thought about it for a bit I now think there is a serious problem for Cartesian epistemology here.
How are we supposed to rule out that we are not in some Job-like situation in which God allows the Evil Demon to decieve us into thinking that there is a physical world (in order to test us or whatever). So even if you grant all of Descartes’ premises you still don’t really have any justification to believe in the existence of the physical world because you can’t rule out this final Evil Demon scenerio (i.e. the one where God allows him to decieve you).