File upload doesnt caching...

  • (3 Pages)
  • +
  • 1
  • 2
  • 3

36 Replies - 4238 Views - Last Post: 25 October 2017 - 12:37 PM

#16 ArtificialSoldier  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 1829
  • View blog
  • Posts: 5,761
  • Joined: 15-January 14

Re: File upload doesnt caching...

Posted 04 October 2017 - 03:23 PM

Quote

I understand what you saying, but i was thinking about FastCGI with and without php.ini

This confuses me, because FastCGI has nothing to do with php.ini. It is literally exactly the same with and without the configuration file for PHP. If we're going to be talking about something we need to use the same terminology. FastCGI is not PHP, it is the piece of software between PHP and the web server. It does not use PHP's configuration file at all, FastCGI has its own configuration file. I'm just saying all of this to illustrate the importance of using the same terminology. If you're talking about PHP, don't call it FastCGI, that's confusing. If I have a problem with my computer, where my browser won't download files or something, I'm not going to tell people that Windows isn't working. Windows and Opera are 2 separate things. If someone asks me about how Windows is configured, and I send them the configuration file for Opera, they're going to be confused. We need to use the same terminology.

Quote

but under eg. Slowloris attack thats bad thing to do, because all FastCGIs can get locked down

"All FastCGIs"? You have no idea what FastCGI actually is, do you? A slowloris attack tries to consume all of a web server's open connections. FastCGI is not a web server connection. Also, slowloris doesn't need to have anything to do with file uploads. You can launch a slowloris attack just with headers. You don't even need a body.

Quote

only with big files it can be noticed

So maybe you're seeing the normal behavior about how the web server handles file uploads, and for some reason you're thinking it's a problem when it's actually just how the server works. The FastCGI is causing problems writing the FastCGI to the FastCGI, because the FastCGI can't CGI fast.
Was This Post Helpful? 1
  • +
  • -

#17 Radius Nightly  Icon User is offline

  • D.I.C Head

Reputation: 6
  • View blog
  • Posts: 91
  • Joined: 07-May 15

Re: File upload doesnt caching...

Posted 05 October 2017 - 07:06 AM

I understand that. I didnt mess with FastCGI settings, its a default one: https://www.hastebin...utuvoxixu.nginx
But because some problems appear after setting php.ini in use i didnt know where the problem is, the closest thing to think about was PHP and FastCGI, not Nginx.

Im using upstream with least_conn, peoples saying if fastcgi_pass are set, Slowloris cant reach FastCGI, but from my tests it does, flooding all FastCGIs, and they become unavailable for anything for some time. It was long time ago, maybe because of other settings that i was messing with at that time, or because Nginx limit connections are above upstream connections or i dont know what, cant test it again (lazy to set it up on Windows again).

Judging from all of this ATM (changing some Nginx settings, specially body caching to drive that didnt work before), i dont think Slowloris will be an issue, but i cant know.
Nginx for Windows got some stupid bugs, i understand why, like you saw already in my config:
pid logs/nginx.pid;
error_log logs\error.log error;
It works. This one:
pid logs\nginx.pid;
error_log logs/error.log error;
Doesnt work. Defaults are usually /, and Windows wont work with them, so after i set up manually client_body_temp_path to temp\client_body_temp (thats default, but with /) it started working. For some of them he say that path doesnt exist, for some he doesnt, or PHP error log will say similar thing for cache files.

Yeah, maybe its not a problem (he is using half a limit for files who exceed limit), but knowing what he is doing at that scenario is helpful and useful to know in my position (wont test other versions to see what they do).
Was This Post Helpful? 0
  • +
  • -

#18 ArtificialSoldier  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 1829
  • View blog
  • Posts: 5,761
  • Joined: 15-January 14

Re: File upload doesnt caching...

Posted 05 October 2017 - 10:32 AM

Maybe you just didn't notice any issues until you changed the default values for PHP. Every option in PHP has a default value, so if you weren't using a configuration file to start with, that just means that everything was default. This is the list of options and their default values:

http://php.net/manual/en/ini.core.php

The default max upload filesize is only 2M, and the max post size is only 8M. But the reason why I don't think this has anything to do with PHP is because PHP doesn't even start while the upload is in progress, it only starts after the entire request has been received by the web server. In the case of a slowloris attack, for example, PHP would never even run. Slowloris is an attack against the actual web server, not other software that it's running.

Quote

Slowloris cant reach FastCGI, but from my tests it does, flooding all FastCGIs, and they become unavailable for anything for some time.

That doesn't make any sense. Your web server has exactly 1 copy of FastCGI. Again, you need to figure out which terminology to use. If you're saying that your server has more than one FastCGI, then is there also more than 1 FastCGI config file? No, there's only 1.

Quote

Judging from all of this ATM (changing some Nginx settings, specially body caching to drive that didnt work before), i dont think Slowloris will be an issue, but i cant know.

In about 5 minutes I found several resources talking about why slowloris attacks don't work on Nginx, at least the new versions. If you're using a recent version of Nginx then you can limit the number of connections per IP address, and you can also drop requests that deliver the body of a post request too slowly. Neither of those have anything at all to do with FastCGI, or PHP. That is Nginx only, because slowloris does not attack FastCGI, and it does not attack PHP, it attacks the web server itself. Again, you need to learn and use the correct terminology.
Was This Post Helpful? 2
  • +
  • -

#19 Radius Nightly  Icon User is offline

  • D.I.C Head

Reputation: 6
  • View blog
  • Posts: 91
  • Joined: 07-May 15

Re: File upload doesnt caching...

Posted 05 October 2017 - 01:20 PM

Yeah, from default to custom it was. Thanks for the default list.

It didnt make sense, but i guess it was about settings, and i dont think its still the same.
Some bugs has been reported about writing body to temp folder, usually a crash, so i guess i dont have that problems. :D

BTW Should i care about this: https://www.hastebin.../ebigigikaf.log
Its happening at login (control.php), and its actually working fine, cant get same error again and again when i want. Access denied, guess its more like file in use (its generated site), looking at log (their hash), looks like same files are repeating, like trying to overwrite it with a new one (the same one) while its still in use. Searching around, peoples say if im using fastcgi_cache_lock (thats 2sec), i should disable open_file_cache (300sec), but both are useful. Will try several times more meantime, probably nothing bad, but i dont like Nginx when reporting any kind of "critical" error.
Was This Post Helpful? 0
  • +
  • -

#20 ArtificialSoldier  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 1829
  • View blog
  • Posts: 5,761
  • Joined: 15-January 14

Re: File upload doesnt caching...

Posted 05 October 2017 - 02:52 PM

Quote

BTW Should i care about this

You should care about any error message. At a minimum, it might be an indication of something wrong somewhere else. One of those file paths uses a different slash, maybe that's an indication of a configuration problem.
Was This Post Helpful? 1
  • +
  • -

#21 Radius Nightly  Icon User is offline

  • D.I.C Head

Reputation: 6
  • View blog
  • Posts: 91
  • Joined: 07-May 15

Re: File upload doesnt caching...

Posted 05 October 2017 - 04:20 PM

I disable open_file_cache, and no errors, for now, but i will leave it for some longer time, testing it from time to time. Guess its a problem with this, because i notice before on FTP in my experimental folder (where i put lots of garbage for test), some files become locked (not a problem for any site, but for testing it is), usually index.php, home.php or similar and cannot be deleted via FTP, or can be deleted after a long time (like try tomorrow), after checking it out, was unable to delete them because they are in use, Unlocker pop up and show that files are used by Nginx (can unlock it and delete it; not smart thing to do on actual server). I mean access denied sounds like user privileges issue (i dont mess with more users), there is no way it can be the problem, and can be that file are in use by some process. If slash are the problem (yeah, they are different depending on command, because in other way they wont work, probably Unix vs Windows thing), i usually get error about the wrong path, unavailable directory, missing file and that kind of errors, or none. Nginx like to write error by OS, like common 10055 TCP error on Windows "An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full" when 1024 connections are eaten, and since Error 5 for Windows also means that access are denied (usually fix are to run as administrator), that cant be the case on old XP with Administrator user only who got everything in place (where run as admin doesnt make difference), specially because its running as a system service. Will report if i get any error, if not, then i may just try lowering open_file_cache times or increasing fastcgi_cache_lock times (no, no, its bad, ok, ok, IDK what then : D ).

BTW Off Topic - I hate when i hit "post", and i get to http://www.dreaminco...orums/index.php (and there is no post, going back and all i wrote are lost), im doing copy all the time. Am i the only one experiencing this?
Was This Post Helpful? 0
  • +
  • -

#22 Radius Nightly  Icon User is offline

  • D.I.C Head

Reputation: 6
  • View blog
  • Posts: 91
  • Joined: 07-May 15

Re: File upload doesnt caching...

Posted 07 October 2017 - 05:10 PM

OK, i figure out, its stupid, open file cache was locking it for longer time (300s and 360s) then fastcgi/proxy (2s and 20s), so when each of them try to modify an expired file (2s/20s expire; like after 2-3min), cant because its locked by open file (300s), so i match same times for all of them:
open_file_cache, proxy_cache_path, fastcgi_cache_path (inactive time)
and
open_file_cache_valid, proxy_cache_valid, fastcgi_cache_valid (time)

Default fastcgi/proxy cache are like 5sec (i left lock at 2sec), but have to avoid interruption with open file cache (who wont allow newer file to be generated for 5min). Guess its nothing to care about?
Was This Post Helpful? 0
  • +
  • -

#23 Radius Nightly  Icon User is offline

  • D.I.C Head

Reputation: 6
  • View blog
  • Posts: 91
  • Joined: 07-May 15

Re: File upload doesnt caching...

Posted 09 October 2017 - 06:19 PM

OK, i got another issue now. Actually two.

1. New visitor stuck at waiting for mydomain.com at first visiting only.
Like, you open the site for the first time, connecting to DNS, connected, you get title, background picture, text, style, font, etc., and for some reason it stuck at php, leaving status in your web-browser "Waiting for mydomain.com..."
On the server side, he did everything, didnt left any error. For him, client got it all right, and server just keep alive connections, nothing else to do, but client are stuck at waiting, and can wait forever. If your web-browser timeout are set to 60sec, well after your timeout, site will be loaded (will take 60sec). My web-browser timeout are 5min, if i set it to 5sec, first visiting will take 1sec+5sec to load (because my browser wont wait longer then 5sec in the end).
So if you hit F5 to refresh the site (while waiting), or CTRL+F5 for clear refresh to load everything as a new, everything is gonna be loaded really fast, and you wont stuck at waiting, only at the first visiting. That can be frustrated for anyone.
PopADS has the longest load on tests, and its known to be slow, but i dont think its about him, because client will get status waiting for popads.net, or something like that.
This looks like my server didnt send the content-length (but he does according to curl), or the last byte, or something like that, so visitors web-browser are left to wait for nothing (and will wait until visitors web-browser timeout or if he hit refresh page).
Tried removing max_ranges option because it doesnt send content-length for downloads bigger then his value, but he only works on static content, without PHP, so this didnt help. Also tried replacing 308 with 301 and removing redirect from http to https.

2. https://www.hastebin.../uqihurakih.log
This is upload (caching body to a disk) and download (caching from FastCGI to serve), thats warning, nothing to care about, it works fine as you can see, this thread was about that. But at the end of download (only when download are complete, nowhere else) im getting critical error in log "SSL_write() failed (SSL:)" and error from Windows OS "10053: An established connection was aborted by the software in your host machine" that can be connected to anything (like when someone say, my PC are not working, whats the problem?), cant know. There is no firewall or similar things to block anything. And from the client side, everything are normal, CRC from same file on server and on my laptop are identical, and client can download lots of files, at each download complete, error log will be longer for one line of "SSL_write() failed (SSL:)..." Nothing special i guess, but its annoying, and the first problem should be at higher priority then this.
Was This Post Helpful? 0
  • +
  • -

#24 ArtificialSoldier  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 1829
  • View blog
  • Posts: 5,761
  • Joined: 15-January 14

Re: File upload doesnt caching...

Posted 10 October 2017 - 10:21 AM

If things are taking a long time to load, instead of guessing at the problem use your browser's developer tools to look at all of the requests and figure out which one is still open. You don't have to guess about the headers either, look at the request and response headers in the developer tools if you want to see them.

For the other error message, there's a lot of information online about that, so I would start researching there.
Was This Post Helpful? 1
  • +
  • -

#25 Radius Nightly  Icon User is offline

  • D.I.C Head

Reputation: 6
  • View blog
  • Posts: 91
  • Joined: 07-May 15

Re: File upload doesnt caching...

Posted 10 October 2017 - 01:05 PM

Thanks, good starting point, didnt came on my mind.

This is when it stuck, its functions.js.php (i stop it at 8th sec).
Posted Image

This is when load are normal.
Posted Image

When it stuck, usually functions.js.php got blocked in 1ms, and thats it.
Posted Image

Longest load has PopADS, but even he loads in the end.
Posted Image

And error are shown for functions.js.php.
Posted Image
With easy fix: https://developer.mo...Unexpected_type

:tup:
Was This Post Helpful? 0
  • +
  • -

#26 ArtificialSoldier  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 1829
  • View blog
  • Posts: 5,761
  • Joined: 15-January 14

Re: File upload doesnt caching...

Posted 10 October 2017 - 04:27 PM

That error message means the Javascript is trying to get an element that doesn't exist and do something with it. That doesn't have anything to do with load times though.
Was This Post Helpful? 1
  • +
  • -

#27 Radius Nightly  Icon User is offline

  • D.I.C Head

Reputation: 6
  • View blog
  • Posts: 91
  • Joined: 07-May 15

Re: File upload doesnt caching...

Posted 10 October 2017 - 05:54 PM

Yeah, it exists after the upload, when links are generated, for copy link to clipboard buttons. I just put the whole function in comment to see if thats a problem.
In the end fastcgi_keep_conn off fixed the long waiting (it was on), also put fastcgi_ignore_client_abort off and fastcgi_cache_methods GET HEAD, all of that are defaults, but i like to have it written.

I think i know whats SSL errors in log, but not sure, was testing with ssllabs.com who get 3 SSL errors in my log for each test. Im using TLS 1.0, 1.1 and 1.2. TLS 1.3, SSL 2 and 3 are disabled in config. When web-browser try to use something else, im getting SSL error in error.log, only on GET requests ATM on direct download links.
As you notice, i dont like error.log to have stupid errors. Warnings when he is caching and that kind of can be disabled (because i know he is gonna do that), but errors above warning are, error, critical, alert and emergency, that sounds like some fatal issue to me, its stupid if this is a case why im getting critical SSL errors (i understand, SSL, blah, blah, but TLS 1.3, SSL2 and SSL3 are disabled, i dont care if someone trying to use that). :/

This post has been edited by Radius Nightly: 10 October 2017 - 05:56 PM

Was This Post Helpful? 0
  • +
  • -

#28 Radius Nightly  Icon User is offline

  • D.I.C Head

Reputation: 6
  • View blog
  • Posts: 91
  • Joined: 07-May 15

Re: File upload doesnt caching...

Posted 14 October 2017 - 06:08 PM

Hi again.

I notice some strange problem on download. Happening on download zip files, via PHP or static, doesnt matter whats inside zip, means have to fix it in Nginx, and its happening only on zip files under 10MB. When you try to download any zip under 10MB, your download gets on download list, first you will get download information in your web-browser "This file is not commonly downloaded.", to start downloading, you have to right click it and select Allow Download, you get information: "This file is not commonly downloaded and may not be safe to open. It may contain a virus or make unexpected changes to your programs and settings. You can search for an alternate download source or try again later.", you accept it, and download file are fine.

Looks like content type issue, maybe because of POST or HTTPS? Dont have add_header Content-Type defined, default_type is application/octet-stream. For zip files it should be application/zip, but Nginx for ZIP files under 10MB sending content-type "application/x-www-form-urlencoded", and there is no such type written anywhere (any site, any config, any php, mime.types, nowhere, i search for it with AstroGrep).

Web-browser sending wrong request (probably because of HTTPS), so i have to define content-type to match all kind of them, right? Nginx sending response header with application/octet-stream, not zip. How should i define it to keep it simple stupid, so Nginx can know what type he is sending? Mime types are from IANA, all kind of them. https://www.hastebin...visivafoc.nginx

This is example what i get from web-browser: https://www.hastebin.../raw/usezolidur

Thanks.
Was This Post Helpful? 0
  • +
  • -

#29 ArtificialSoldier  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 1829
  • View blog
  • Posts: 5,761
  • Joined: 15-January 14

Re: File upload doesnt caching...

Posted 16 October 2017 - 09:42 AM

Have you researched the actual error message to see what other people are saying about it?

The correct mime type for a zip file should be application/zip.
Was This Post Helpful? 1
  • +
  • -

#30 Radius Nightly  Icon User is offline

  • D.I.C Head

Reputation: 6
  • View blog
  • Posts: 91
  • Joined: 07-May 15

Re: File upload doesnt caching...

Posted 18 October 2017 - 07:35 AM

Peoples saying its by Google and i can contact them to fix it, but i guess its more like Firefox, Chrome, and others web browser security check thing. Usually peoples say that kind of issue comes by itself and goes by itself. I can take troll_face.jpeg, put it in the zip and upload it, on download i will get that warning message by web-browser. Nginx works fine, sending ZIP files with application/zip, and problem appear by itself (i tough i made a problem), first ZIP files of 12.5MB works fine, ZIP of 10MB and less got warning, ATM (i did nothing) looks like ZIP of 8MB and less has warning. Because its HTTPS, and probably for some security reasons in web-browser, he is sending request for content (like ZIP under 10MB) to be application/x-www-form-urlencoded type of content, guess its just another layer of security from web-browser.
Im using Comodo DNS for security and not my poor ISP DNS. Im not talking about DNS for web site domain and IP, thats DtDNS. He is filtering all activity and blocking bad things trough connections. I mean, he usually block things like Facebook or so for several minutes if someone on that DNS try/gain something potentialy bad, giving you known or unknown warning, or options to create a virtual network for several minutes if you allow that access or that kind of stuff, he usually allows or block, nothing in between, but could that be the problem?
Didnt test with password protected ZIP (probably will be the same, if jpg or single gif in zip representing some kind of threat). Didnt test on another network (phone or laptop connected to another internet connection). And last thing i wanna test is DNS, because here ISP DNS are a really bad idea for anyone, i deal with so many computers infected by it, from government, to gain or steal more money from citizens.
Was This Post Helpful? 0
  • +
  • -

  • (3 Pages)
  • +
  • 1
  • 2
  • 3