@elam84
Then I do the 3 sets of max added up and divide by 3. So e.g. I do 10, 8, 6 = 24/3 = 3x8 I should aim for. Repeat till 3x12 then go up weight.
I'm rather confused here.
Your best option is to follow a proven routine that will lay out the progression scheme for you.
But to explain how I'd go about progression if you were to jsut keep going... have your goal rep range, whatever it may be. Lets say its 10-12 reps. Your first few workouts may just be playing with weights to figure out where you need to be. But lets say you settle on 25lb dumbbells for this example. You manage to get 10, 10, 10. So next time you lift, you hit 12, 10 10. Maybe after that, you hit 12, 12, 12. If you super struggled that last rep, you could maybe do another workout of 3x12, or you could try moving up to the 30lb dumbbells. Now lets say with the 30s you only manage 8, 8, 8. That's fine, keep pushing and continue inching your way up.
If you manage then something like 12, 8, 8... you may want to have a look at your rest time. Resting a bit longer may give you just a bit more strength to hit higher reps.
Now lets say you jumped up to 30 and your reps tanked at like 5, 5, 5. This happens for some exercises when the jump in weight is too much. What I like to do in this case is to do those 5 reps and then immediately drop down to the 25s and finish out the set. This is called a drop set. After your rest, you grab those 30s again and repeat.
There also is merit to mixing up rep counts and rest times as well. My routine has some workouts be 13-15 reps, while others are as low as 3-5 (usually I reserve those for barbell work). The rest times also vary from a minute to 3 minutes. I'm gonna be able to lift heavier weights across any rep range with a longer rest time. But a shorter rest time gets you to failure quicker. But you still want to get enough volume. So this is why I leave this kind of detail to a routine written by someone smarter than me.
https://thefitness.wiki/muscle-building-101/
Also diet is incredibly important. Not eating enough protein will hinder everything. If you're also trying to lose weight (ie, calorie deficit) you'll also struggle a bit, but that's accepted since the goal is weight loss at that point.