One of the standards on the subject is Experimental Composite Propellant by Terry McCreary.
For best results, I'd suggest picking up a copy of Terry's book, and reading it. In addition, find someone in your area who has a proven record with making motors, and spend time working with that person.
Also, understand that just because you can successfully make a 38mm or 54mm motor doesn't mean you can make a 98mm that will work. Likewise folks who can make 98s can't necessarily make larger than 98mm.
Each step along the way is a new learning curve.
-Kevin