-set it as a new ability and add as necessary
You mean make an aura with a flat damage bonus and specifically set each level to be XX number (that I already determined) and then give the hero that aura at level XX? Seems inefficient and I don't even understand what you are saying anyways.
-Calculate a new ability strength to add [(total damage - base damage (assuming its constant)) * .5] + [(Total Damage)]
I don't think I can calculate total/base damages without hitting a unit... meaning I'd have to hit someone before my skill would "reset" to its actual level. Because I'm sure you can't add that formula to a regular spell lol. Also for this one you would have to create another aura or passive item spell and add it to the unit.. and you cannot set a random number to be the amount of damage the aura/item does. It has to be set within the world editor before hand, which means once again... inefficient.
If I misread something inform me.