Hi @Manasi29
One approach to add styling to your Accordion is that you can add your own custom sass. For the icons, there are two default classes for normal and expanded view button, you can add icons there. It's better to use your own custom sass.
.cmp-accordion__button {
content: <here you can add either URL of icon using url(" ") or the variable which you have already defined in your variables sass>;
}
.cmp-accordion__button--expanded {}
Defining variables is better approach.