Skocz do zawartości

Web Hosting Talk

  • progreso.pl

    Partner technologiczny

    Upraszczamy to, co inni starają się komplikować. Prosto, pewnie, przyjaźnie - tak robimy hosting!
  • Kei.pl

    Partner technologiczny

    Kei.pl działa na polskim rynku internetowym od 2000 roku. Obecnie na blisko 300 serwerach w Centrum Danych Kei.pl znajduje się kilkadziesiąt tysięcy stron WWW.
  • S-NET.info

    Partner technologiczny

    S-NET to dostawca usług dla biznesu. Najważniejsze usługi świadczone przez firmę to usługi Centrum Danych, dostęp do Internetu, transmisja danych oraz tranzyt do różnych operatorów.
  • Sprint Data Center

    Partner technologiczny

    Sprint Data Center to jedyne w Polsce północno-wschodniej i jednocześnie jedno z najnowocześniejszych w kraju centrum przechowywania i przetwarzania danych.

 

andycole

Dołączył: 26 mar 2006
Offline Ostatnio aktywny: kwi 23 2012 02:16

Moje tematy

nginx+php_fpm+mysql pomoc w optymalizacji configu

31 marzec 2012 - 15:09

Witam,

Mój VPS to:
2x Intel® Xeon E5520 @ 2.27GHz
4 GB RAM

Próbuję jak najoptymalniej go skonfigurować, ale miotam się trochę jak dziecko we mgle. Poszukuję jakiejś dobrej duszy, która pomoże w zoptymalizowaniu konfiguracji serwera albo dobrego specjalisty, który pomoże profesjonalnie - odpłatnie.

Ustawienia mysql:

[mysqld]
collation_server=utf8_unicode_ci
character_set_server=utf8
skip-character-set-client-handshake
max_connections=70
max_user_connections=30
read_buffer_size = 8M
read_rnd_buffer_size = 8M
sort_buffer = 16M
join_buffer_size = 2M
key_buffer = 256M
query_cache_size = 32M
max_heap_table_size = 256M
tmp_table_size = 256M
max_allowed_packet = 128M
table_cache = 1024
### Global variables ###
thread_stack = 512K
thread_cache_size = 384
wait_timeout = 80
myisam_sort_buffer_size = 64M
binlog_cache_size = 65536
max_join_size = 8M
thread_concurrency = 4
query_cache_limit = 3M
query_cache_type = 1
query_cache_min_res_unit = 1024
#low_priority_updates = 1
long_query_time = 1
log_error = error.dat
log-slow-queries = slow.dat
log-queries-not-using-indexes
skip-innodb
skip-name-resolve
skip-bdb

nginx

worker_processes  2;
#worker_rlimit_nofile 3000;
user apache;
events{
	worker_connections  1024;
}
http{
	include mime.types;
	default_type application/octet-stream;
server_tokens off;
access_log off;

open_file_cache max=2000 inactive=1m;
open_file_cache_valid 1m;
open_file_cache_min_uses 3;
open_file_cache_errors on;

fastcgi_buffer_size 4k;
fastcgi_buffers 256 4k;
fastcgi_max_temp_file_size 0;

#proxy_buffers 256 4k;
#proxy_max_temp_file_size 0;
	gzip on;
	gzip_min_length 2000;
	gzip_types text/plain text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript;

include /usr/local/nginx/conf/vhosts/*;
}

php fpm

emergency_restart_threshold 10
emergency_restart_interval 1m
process_control_timeout 5s
daemonize yes
max_children 25
StartServers 10
MinSpareServers 5
MaxSpareServers 20
request_terminate_timeout 15s
request_slowlog_timeout 5s
rlimit_files 0
rlimit_core 0
max_requests 600

memcached ustawiony na 128mb, zużycie na poziomie 3%

zrzut z iostat:

Linux 2.6.18-194.8.1.el5.028stab070.5PAE (vps)  03/31/12  _i686_
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
		   3.63	0.00	1.60   18.47	0.00   76.30

Load utrzymuje się na poziomie ok 1. Na liście procesów nie widać jakiegoś specjalnie dużego obciążenia, ale iowait skacze i to wysoko.


Status mysqla zwraca mi:

[/left]
[left]Created_tmp_tables 351
Created_tmp_disk_tables 64
Open_tables 387
Opened_tables 393

Podejrzewam, że wina leży właśnie po stronie mysqla - tworzone są tabele na dysku. Tylko jak sobie z tym poradzić? Wydaje mi się że

max_heap_table_size = 256M
tmp_table_size = 256M
table_cache = 1024

są wystarczające.... Jak uważacie?


Może dałoby radę też zoptymalizować ustawienia nginx/php fcgi dla większej wydajności?

Pozdrawiam
andycole

optymalizacja mysql, wolne zapytania

20 marzec 2012 - 00:44

Witam,

Mam problem z VPSem. Config to:

2x Intel® Xeon E5520 @ 2.27GHz
80 GB HD SAS RAID1
4 GB RAM

Mysql 5.3, php 5.3 fcgi z xCache, nginx, memcache.

Problem, polega na tym, że load jest na poziomie 1-2, mimo że na liście htop jest tylko kilka procesów po ok +/- 1% zużycia CPU. Dodatkowo co jakiś czas do slow queries log zapisywane są bardzo proste zapytania, których czas wykonania oscyluje między 7, a nawet 40 sekund! Często zdarza się, że serwer odpowiada na ping, ale serwis nie działa (mysql i prawdopobnie ssh wiszą). Jest to bardzo irytujące i nie mam już pomysłu jak sobie z tym radzić.

Wiem, że minęło dopiero 8 godzin od startu serwera, ale mam nadzieję, że uda Wam się mi pomóc mimo to. Mam stosunkowo dużo zapytań zapisujących do bazy, 3 gb wolnego ramu... Wiem, że niektóre bufory strasznie duże, ale to efekt mojej walki, metodą prób i błędów Dodany obrazek Poniżej zrzuty ze skryptów:

mysqlreport:

MySQL 5.0.67-log		 uptime 0 7:55:53	   Mon Mar 19 23:51:53 2012
__ Key _________________________________________________________________
Buffer used	 9.70M of 512.00M  %Used:   1.89
  Current	  67.51M			%Usage:  13.18
Write hit	  27.66%
Read hit	   99.51%__ Questions ___________________________________________________________
Total		   1.04M	36.6/s
  Com_		373.67k	13.1/s  %Total:  35.79
  QC Hits	 369.90k	13.0/s		   35.43
  COM_QUIT	187.93k	 6.6/s		   18.00
  DMS		 113.67k	 4.0/s		   10.89
  -Unknown	  1.21k	 0.0/s			0.12
Slow 1 s		2.63k	 0.1/s			0.25  %DMS:   2.32  Log:  ON
DMS		   113.67k	 4.0/s		   10.89
  INSERT	   61.43k	 2.2/s			5.88		 54.05
  SELECT	   29.23k	 1.0/s			2.80		 25.71
  UPDATE	   22.68k	 0.8/s			2.17		 19.96
  DELETE		  322	 0.0/s			0.03		  0.28
  REPLACE		   0	   0/s			0.00		  0.00
Com_		  373.67k	13.1/s		   35.79
  set_option  186.74k	 6.5/s		   17.89
  change_db   186.49k	 6.5/s		   17.86
  show_variab	 163	 0.0/s			0.02
__ SELECT and Sort _____________________________________________________
Scan			2.26k	 0.1/s %SELECT:   7.74
Range			 655	 0.0/s			2.24
Full join		   0	   0/s			0.00
Range check		 0	   0/s			0.00
Full rng join	   0	   0/s			0.00
Sort scan		 494	 0.0/s
Sort range		955	 0.0/s
Sort mrg pass	   0	   0/s__ Query Cache _________________________________________________________
Memory usage	3.65M of  64.00M  %Used:   5.71
Block Fragmnt  25.27%
Hits		  369.90k	13.0/s
Inserts		27.53k	 1.0/s
Insrt:Prune  27.53k:1	 1.0/s
Hit:Insert	13.44:1
__ Table Locks _________________________________________________________
Waited			283	 0.0/s  %Total:   0.22
Immediate	 129.41k	 4.5/s__ Tables ______________________________________________________________
Open			  388 of  512	%Cache:  75.78
Opened			501	 0.0/s
__ Connections _________________________________________________________
Max used		   39 of   70	  %Max:  55.71
Total		 187.94k	 6.6/s__ Created Temp ________________________________________________________
Disk table		104	 0.0/s
Table			 714	 0.0/s	Size:  64.0M
File				0	   0/s
__ Threads _____________________________________________________________
Running			 1 of	1
Cached			 38 of  384	  %Hit:  99.98
Created			39	 0.0/s
Slow				0	   0/s__ Aborted _____________________________________________________________
Clients			36	 0.0/s
Connects		1.08k	 0.0/s
__ Bytes _______________________________________________________________
Sent		  489.71M   17.2k/s
Received	   64.48M	2.3k/s__ InnoDB Buffer Pool __________________________________________________
Usage			   0 of	   0  %Used:   0.00
Read hit		0.00%
Pages
  Free			  0			%Total:   0.00
  Data			  0					  0.00 %Drty:   0.00
  Misc			  0					  0.00
  Latched		   0					  0.00
Reads			   0	   0/s
  From file		 0	   0/s			0.00
  Ahead Rnd		 0	   0/s
  Ahead Sql		 0	   0/s
Writes			  0	   0/s
Flushes			 0	   0/s
Wait Free		   0	   0/s
__ InnoDB Lock _________________________________________________________
Waits			   0	   0/s
Current			 0
Time acquiring
  Total			 0 ms
  Average		   0 ms
  Max			   0 ms__ InnoDB Data, Pages, Rows ____________________________________________
Data
  Reads			 0	   0/s
  Writes			0	   0/s
  fsync			 0	   0/s
  Pending
	Reads		   0
	Writes		  0
	fsync		   0
Pages
  Created		   0	   0/s
  Read			  0	   0/s
  Written		   0	   0/sRows
  Deleted		   0	   0/s
  Inserted		  0	   0/s
  Read			  0	   0/s
  Updated		   0	   0/s

mysql-tuner

>>  MySQLTuner 1.0.1 - Major Hayden <major@mhtx.net>
>>  Bug reports, feature requests, and downloads at http://mysqltuner.com/
>>  Run with '--help' for additional options and output filtering-------- General Statistics --------------------------------------------------
[--] Skipped version check for MySQLTuner script
[OK] Currently running supported MySQL version 5.0.67-log
[!!] Switch to 64-bit OS - MySQL cannot currently use all of your RAM
-------- Storage Engine Statistics -------------------------------------------
[--] Status: -Archive -BDB -Federated -InnoDB -ISAM -NDBCluster 
[--] Data in MyISAM tables: 95M (Tables: 254)
[!!] Total fragmented tables: 5-------- Performance Metrics -------------------------------------------------
[--] Up for: 8h 32m 46s (1M q [35.169 qps], 193K conn, TX: 503M, RX: 67M)
[--] Reads / Writes: 24% / 76%
[--] Total buffers: 650.0M global + 35.5M per thread (70 max threads)
[!!] Allocating > 2GB RAM on 32-bit systems can cause system instability
[!!] Maximum possible memory usage: 3.1G (78% of installed RAM)
[OK] Slow queries: 0% (2K/1M)
[OK] Highest usage of available connections: 55% (39/70)
[OK] Key buffer size / total MyISAM indexes: 512.0M/58.6M
[OK] Key buffer hit rate: 99.5% (2M cached / 14K reads)
[OK] Query cache efficiency: 92.5% (379K cached / 409K selects)
[OK] Query cache prunes per day: 0
[OK] Sorts requiring temporary tables: 0% (0 temp sorts / 1K sorts)
[OK] Temporary tables created on disk: 10% (139 on disk / 1K total)
[OK] Thread cache hit rate: 99% (39 created / 193K connections)
[OK] Table cache hit rate: 77% (390 open / 503 opened)
[OK] Open file limit used: 59% (662/1K)
[OK] Table locks acquired immediately: 99% (143K immediate / 143K locks)
-------- Recommendations -----------------------------------------------------
General recommendations:
	Run OPTIMIZE TABLE to defragment tables for better performance
	MySQL started within last 24 hours - recommendations may be inaccurate

mysql-tuning-primer


(BUsing login values from /.my.cnf
(B- INITIAL LOGIN ATTEMPT FAILED -(BTesting for stored webmin passwords:(B None Found
(BCould not auto detect login info!(BFound Sockets:
/tmp/mysql.sock
(BUsing: /tmp/mysql.sock
(B
(B
(B (B-- MYSQL PERFORMANCE TUNING PRIMER --
(B	  - By: Matthew Montgomery -
(B
MySQL Version 5.0.67-log i686
(B
Uptime = 0 days 8 hrs 30 min 52 sec
(BAvg. qps = 35
(BTotal Questions = 1080695
(BThreads Connected = 1
(B
Warning: (BServer has not been running for at least 48hrs.
(BIt may not be safe to use these recommendations
(B
To find out more information on how each of these
(Bruntime variables effects performance visit:
(Bhttp://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html
(BVisit http://www.mysql.com/products/enterprise/advisors.html
(Bfor info about MySQL's Enterprise Monitoring and Advisory Service
(B
SLOW QUERIES
(BThe slow query log is enabled.
(BCurrent long_query_time = 1 sec.
(BYou have (B2692 (Bout of (B1080721 (Bthat take longer than 1 sec. to complete
(BYour long_query_time seems to be fine
(B
BINARY UPDATE LOG
(BThe binary update log is (BNOT (Benabled.
(BYou will not be able to do point in time recovery
(BSee http://dev.mysql.com/doc/refman/5.0/en/point-in-time-recovery.html
(B
WORKER THREADS
(BCurrent thread_cache_size = 384
(BCurrent threads_cached = 38
(BCurrent threads_per_sec = 0
(BHistoric threads_per_sec = 0
(BYour thread_cache_size is fine
(B
MAX CONNECTIONS
(BCurrent max_connections = 70
(BCurrent threads_connected = 1
(BHistoric max_used_connections = 39
(BThe number of used connections is (B55% (Bof the configured maximum.
(BYour max_connections variable seems to be fine.
(B
MEMORY USAGE
(BMax Memory Ever Allocated : 1.92 G
(BConfigured Max Per-thread Buffers : 2.42 G
(BConfigured Max Global Buffers : 586 M
(BConfigured Max Memory Limit : 2.99 G
(BPhysical Memory : 3.90 G
(BMax memory limit seem to be within acceptable norms
(B
KEY BUFFER
(BCurrent MyISAM index space = 58 M
(BCurrent key_buffer_size = 512 M
(BKey cache miss rate is 1 : 211
(BKey buffer free ratio = 86 %
(BYour key_buffer_size seems to be fine
(B
QUERY CACHE
(BQuery cache is enabled
(BCurrent query_cache_size = 64 M
(BCurrent query_cache_used = 4 M
(BCurrent query_cache_limit = 3 M
(BCurrent Query cache Memory fill ratio = 6.50 %
(BCurrent query_cache_min_res_unit = 4 K
(BYour query_cache_size seems to be too high.
(BPerhaps you can use these resources elsewhere
(BMySQL won't cache query results that are larger than query_cache_limit in size
(B
SORT OPERATIONS
(BCurrent sort_buffer_size = 7 M
(BCurrent (Bread_rnd_buffer_size (B= 8 M
(BSort buffer seems to be fine
(B
JOINS
(BCurrent join_buffer_size = 4.00 M
(BYou have had 0 queries where a join could not use an index properly
(BYour joins seem to be using indexes properly
(Bjoin_buffer_size >= 4 M
(BThis is not advised
(B
OPEN FILES LIMIT
(BCurrent open_files_limit = 1104 files
(BThe open_files_limit should typically be set to at least 2x-3x
(Bthat of table_cache if you have heavy MyISAM usage.
(BYour open_files_limit value seems to be fine
(B
TABLE CACHE
(BCurrent table_cache value = 512 tables
(BYou have a total of 271 tables
(BYou have (B390 (Bopen tables.
(BThe table_cache value seems to be fine
(B
TEMP TABLES
(BCurrent max_heap_table_size = 256 M
(BCurrent tmp_table_size = 64 M
(BOf 1221 temp tables, 10% were created on disk
(BCreated disk tmp tables ratio seems fine
(B
TABLE SCANS
(BCurrent read_buffer_size = 16 M
(BCurrent table scan ratio = 41 : 1
(Bread_buffer_size is over 8 MB (Bthere is probably no need for such a large read_buffer
(B
TABLE LOCKING
(BCurrent Lock Wait ratio = (B1 : 503
(BYou may benefit from selective use of InnoDB.
(BIf you have long running SELECT's against MyISAM tables and perform
(Bfrequent updates consider setting 'low_priority_updates=1'
(BIf you have a high concurrency of inserts on Dynamic row-length tables
(Bconsider setting 'concurrent_insert=2'.
(B

Zmienne zaznaczone na czerwono ze Status Mysql:

Created_tmp_disk_tables 139
Opened_tables 503
Table_locks_waited 284
Handler_read_rnd_next 1 317 k
Handler_read_rnd 116 k
Slow_queries 2 740

VPS, zwiechy, duży load

23 styczeń 2012 - 00:41

Witam,

Problem wygląda następująco:

Podczas obserwacji poleceniem TOP, load utrzymuje się na poziomie 0.1, 0.2 i nagle zaczyna rosnąć do 4, 5. W tym momencie klient SSH nie odświeża już informacji z polecenia top, serwer przestaje odpowiadać zarówno na ssh jak i porcie 80. Co jest ciekawe, generując obciążenie 4, 5 i więcej na liście procesów poniżej najbardziej zasobożerny jest proces zajmujący jedynie 0.3% CPU. Wszystko kończy się restartem administracji poprzez serwer matkę.

Czy w przypadku wirtualizacji XEN możliwe jest żeby load na moim VPSie generowany był przez serwer matkę lub inne, bliźniacze serwery VPS? Wiem, że pewnie pytanie głupie, ale nie mam już innego pomysłu. Może znacie jakąś inną przyczynę takiej sytuacji?

logi slow query mysql nie zawierają żadnych długich zapytań, logi długich akcji php-cgi również, logi nginx też nie zanotowały nic co mogłoby aż tak zwiesić serwer.

Pozdrawiam