Yes, each segment should contain no more than 20 million unique visitors. Sharing a segment containing more than 20 million will result in unexpected behavior.
If you have a segment that contains greater than 20 million unique visitors, try making the segment more specific or limiting the lookback window. Alternatively, you may be able to create a real-time audience that accomplishes the same goal.
Documentation - Audiences