Agree on the size first
Talk about how many people would make the day feel comfortable for both of you. Check the venue capacity and package tier before sending invitations.
Be clear about partners, children, witnesses and anyone helping with the ceremony so you count consistently.
Make the list together
Write down the people you each want to include and talk through the choices. You do not need a rule based on how recently you saw someone; relationships and circumstances differ.
Keep the agreed list in one place and check the cost and capacity before adding anyone.
Explain the plans kindly
If someone expects an invitation, a clear personal conversation can help. You can explain that you are keeping the ceremony small without ranking the importance of your relationships.
A later meal, call or shared gallery may be a way to include people who will not attend, if that is something you want.
Plan for the guests you choose
Ask about travel, access and dietary needs, and give people a clear timetable. A small wedding still needs comfortable arrangements for everyone there.