I have wp-rocket and same Liyana , I have a large quantity of poor URL in CLS.if you're using wordpress base theme, find a GOOD caching plugin and apply it. I would suggest W3-Total-Cache but it'll need some configurating.
If you site going down, its not because of Core Web Vitals.Sure, I also have more troubles, and my site ranking also fastly going down. I check from Google Search Console, it says, under that Core Web Vitals, Poor ULS 179, in my sites. I checked why it says like that. But, Still, I can't solve it, If you have an idea, please help me to solve it. But, I increased the page loading speed like CLS, LCP for mobile, and Desktops, But still, the problem is there my ranking. What can I do, if you have an idea please help me. Thanks