====== [TROUBLESHOOT] Ceph too few pgs per osd ======

^  Documentation  ^|
^Name:| [TROUBLESHOOT] Ceph too few pgs per osd |
^Description:| how to solve this "issue" |
^Modification date :|13/06/2019|
^Owner:|dodger|
^Notify changes to:|Owner |
^Tags:|ceph, object storage |
^Scalate to:|The_fucking_bofh|


====== WARNING ======

<WRAP center round important 60%>
This documents cover the **TOO FEW** PGs per OSD, not //too many// (documented [[linux:ceph:troubleshooting:too_many_pgs_per_osd|here]])
</WRAP>


====== The error ======

<code bash>
ceph@avmlp-osm-001 ~/ceph-deploy $ sudo ceph health
HEALTH_WARN too few PGs per OSD (4 < min 30)
</code>

Official Documentation: [[http://docs.ceph.com/docs/master/rados/operations/health-checks/#many-objects-per-pg]]
\\
So "the problem" is that there's any pool with less PGs (''pg_num'') that the configured in the warning threshold.

====== The solutions ======

There are 2 solutions for this warning:
  - Increase the ''pg_num'' or ''pg_num_min'' in //autoscale// mode.
  - Change the warning threshold.

===== Increase pg_num/pg_num_min =====

==== pg_num ====
See which pool is the one with less PG's than expected:
<code bash>
ceph osd pool ls detail
</code>

Just change the ''pg_num'', ''pgp_num'' values:
<code bash>
ceph osd pool set ${POOL_NAME} pg_num 500
ceph osd pool set ${POOL_NAME} pgp_num 500
</code>

All pool at once (**TAKE CARE** with this, I've wrote an ''echo'' to avoid problems, think twice before executing):
<code bash>
for i in $(ceph osd pool ls) ; do echo ceph osd pool set ${i} pg_num 60 ; done
</code>

**WARNING** a huge number of PGs in all the pools can lead in error:\\
<code>
Error ERANGE: pool id 4 pg_num 500 size 3 would mean 6000 total pgs, which exceeds max 5000 (mon_max_pg_per_osd 250 * num_in_osds 20)
</code>
See [[linux:ceph:troubleshooting:erange_total_pg_exceeded|[TROUBLESHOOT] Error ERANGE: pool id X pg_num Y size Z would mean W total pgs, which exceeds]]

==== pg_num_min ====

That's for ''pg_autoscale_mode on'':

<code bash>
ceph osd pool set ${POOL_NAME} pg_num_min ${MINPGS}
</code>

If pool actual ''pg_num'' is lower than ''pg_num_min'' you'll have to change ''pg_num'' also:
<code bash>
ceph osd pool set ${POOL_NAME} pg_num ${MINPGS}
ceph osd pool set ${POOL_NAME} pg_num_min ${MINPGS}
</code>

===== Change the warning threshold =====

Add the following to ''ceph.conf'' under ''[global]'' section:
<code>
mon pg warn min per osd = 8
</code>

and restart ''mon'' **and** ''mgr'' (mon to apply the changes, mgr for the monitoring).