Some background
My work-issued laptop is a Windows machine and for the last two years or so, I have had to use PHPStorm which is sort of integrated with our development server(s) and it is fine for the most part. I have tuned it over time to feel more like my preferred text editor; Neovim, with the use of IdeaVim and a couple of keyboard mappings to make in-editor and in-code navigation easier and significantly closer to what I am used to. There was only so much I could do in there to make it any closer to my Neovim + Wezterm setup, but it was still always jarring when I got home, went back to working in Neovim and resumed the next day to go back into PHPStorm.
Also due to how our dev environment is setup, we would often have files out of sync with the Git source and it is not fun to track that down when it happens (yes, your FTP-sniffing senses are correct). This essentially meant that our dev environment did not closely mirror production and always made bug hunting more frustrating since we had to also confirm it wasn’t some file that changed under us in the environment or an actual bug in our new code… but the dev servers themselves are closely configured to match production, and this whole situation just makes that a wasted effort!
What I tried
There had to be a way to make this workflow better. There had been discussions in passing about moving development to the server(s) directly with our laptops becoming thin clients, this sounded good to me as I have had similar workflow in the past and still do with my iPad Air (I will write about that setup later), it was time to explore that.
Fleet
I decided to move to just working directly on the server to remove the upload latency and the files going out of sync, and it seemed like a great opportunity to finally try out Jetbrain’s new editor (Fleet) that has been advertised a lot as useful for remote development, it wasn’t the best experience but this was (and still is) beta software and they ship lots of updates every week, I did not expect it to be feature complete and on par with the older PHPStorm but the Vim mode emulation was completely unusable for me since they had to remake one from scratch and it was missing a lot - as I understand it, it uses an entirely new plugin system and IdeaVim is not supported?
If you are wondering why I did not try PHPStorm itself first for remote development, I completely forgot it had that feature even though I had seen it earlier, I haven’t seen the “splash screen” in a long while now, but the feature was boldly highlighted on Fleet’s homepage that I had been on a couple of times recently.
PHPStorm (Remote Development)
After that whole thing, I realised PHPStorm did support remote development and could install its own binary on the target server, so I proceeded to use that, most of my settings got messed up and I had to reconfigure them and that worked okay once again, I did not entirely love it but it was usable.. pfft.
Neovim (via SSH/Mosh)
I had to edit a commit message and my default editor was a weird version of Vi (I cannot remember what it was called but the server only had that, and Nano), my boss noticed it did different things to what I expected and remembered I was a Neovim user and offered to install Neovim on the server for me instead (I do not have install permissions), I accepted instantly, this was it… I could finally use Neovim directly in my preferred terminal emulator (that I finally now got to use Bash too); Wezterm. I couldn’t import my personal config since it has a lot of dependency on a few things like the Go compiler, Rust, a C compiler, etc to support all the different things I do in there, so I wrote one from scratch loosely based on kickstart.nvim and that’s been pretty… fine. I also added Mosh and Zellij to the mix about two weeks ago and it has been a nice environment to work in!
More context for the switch
My reason for wanting to switch to Neovim at work wasn’t because I am a “Neovim best, other editors bad” kind of person (although I do love Neovim and I think you should try it out), I like my terminal workflow which I have really tuned to be quite personal (jumping between my editor and the terminal without much thought or movement, having my sessions stored as they were regardless of what branch I was on or what directory I was in and ready to go instantly! etc.) but I knew I had to put up with PHPStorm since that is what everyone else was using and to be fair, I kind of like it and what it has to offer out of the box, but it was annoyingly slow(-er); like showing errors for seconds after I had fixed them, taking a while to become responsive every time I switched branches and it had to update its indexes (which I did a lot because I haven’t bothered to learn git worktrees yet), and other smaller issues like just moving between files (I am really used to telescope and ripgrep!). I do understand why it has to do some of these things because I have seen, used and like the features it offers, and I did try some suggested fixes on the internet (feeding it more RAM, clearing the cache so it can rebuild it for some reason and other things) but they did not make any real difference.
I asked some other people in the office and apparently no one else thought it was slow, some folks on Twitter also said they were fine with the performance - perhaps I am the problem and it’s not… slow?
Our repos are fairly large and we share code a lot, which means I usually have multiple projects open, switching branches often and stuff like that, my Neovim setup handles that really well, I would say “but to be fair it is also on a more powerful machine, compared to my work machine” except my work machine is rather decently spec-ed and I have had similar experience with Goland, Webstorm and PHPStorm on my personal daily-driver; the 2021 14” M1 Pro Macbook Pro (same RAM as my work machine; 16GB) with just about two or three windows/workspaces open.
And oh, Jetbrains had 20GB of cache on my personal machine which I just cleared today after getting rid of the last Jetbrains IDE I had left (Android Studio). This is not a campaign against Jetbrains, I really like their products and I have been using them for a long time, they just did not work for me anymore and I am perhaps too poor to buy the ideal machine they will run extremely well on, if they can ever.
The problem
As I mentioned earlier, my work machine runs Windows, but the dev servers obviously run an enterprise distribution of Linux I shall not name. The problem first surfaced with a commit I made for a task, I was inspecting the diff again on the pull request page and saw something strange like this:
-} // This is the last line of the file
+} // This is the last line of the file
\ No newline at end of file
This made no sense to my brain at first, I completely forgot about the OS differences because PHPStorm managed tabs, new lines etc., for me transparently since the client was technically still on Windows (even though the IDE server was on a Linux machine), so I shrugged it off as a UI bug in the version control platform we use, created the pull request anyway, and carried on.
I know you are yelling at your screen right now; “Just tell me the goddamn problem and get to the solution”, we’re getting there, I promise!
Yes, that came back to bite me in the behind, because I had dismissed it earlier, I forgot about it for a short period… that was until I got a comment from my boss asking me to fix it and be careful about whatever my editor was doing, apparently, it’s been in other commits and I simply hadn’t noticed.
What was going on exactly?
I had a proper look with git show in the terminal and oh… there it was, the sneaky ^M. Ah! It was the DOS file format and CR all along! Hang on, let me explain that. You see, Windows represents new lines with \r\n instead of UNIX’s regular \n, and different applications account for that in different ways.
\ris the “carriage return” (CR) character essentially telling the computer “this is the end of this line, move the cursor back to the beginning”, sort of like how typewriters worked back in the day (or so I’ve heard, I never paid much attention to my Mom’s typewriter when she had it) and\nis the “line feed” (LF) - also commonly known as the “new line” - character, that combination Windows uses is also calledCRLF(Carriage Return/Line Feed)
Vim (and by extension; Neovim) here added the ^M character to the end of the file to indicate the carriage return (\r) on that last line (meaning “this is the end of this line, move the cursor back to the beginning”), but not the new line one (LF or \n) because there was indeed no newline there, it correctly detected the file as dos and just did what dos did.
This was partially right but I had the wrong diagnosis here, sort of. Scroll down to read the latest update which contains more accurate information.
What I tried
I initially thought, “Okay, I will just set file format to unix and that’ll be it” forgetting that it will try to convert the whole thing. Running :set fileformat=unix and :w was definitely not a good idea, I ran a hard git reset to try again. I added dos and unix to the file formats list thinking it will figure out the difference and fix it (looking back now, that was also silly; that is not what that does), ran :update and… of course, that did not do it. I tried replacing the invisible PITA character many ways but that also either did not work or broke the whole file somehow… I thought about using Vim to just convert from DOS to Unix format but you can already see the problems:
- It will convert the whole file, not just the last line
- It will cause a lot of unnecessary changes in the diffs
- It will be reverted the next time one of my colleagues edits the file on their Windows machine
This was made worse by the fact that I couldn’t just set the eol config globally as it also caused a lot of other problems in other files, I had to find a way to fix this on a per-file basis as I worked on them. During my search for a solution, I found answers that suggested using :set binary noeol and other things, that was a horrible idea, it turned it all red after rewriting the whole line to strip it all out, don’t do it!
You can see my severe skill issues at play here, while I did sort of know what the problem was, I was having a bit of a trouble figuring out how to fix it without causing more problems.
Ours being a fairly old codebase, it had a mix of a lot of conventions when it came to tabs, expanding tabs etc and things like that but PHPStorm handled most of that nicely for me, I got sleuth to handle most of that for me and it isn’t an issue anymore it seems. Perhaps I should have endured PHPStorm, how I work in the terminal is just quite burned into my brain now and this is the first time I have had to worry about what OS the other person was using.
The (hacky) solution I (initially) settled on
I ended up on doing the following to remove the EOL (end of line) trailing character from the last line of the file (but it is a temporary solution):
:set noeol nofixeol
" :update You may need to do this for some odd reason
:w
I would go as far as making this an autocmd or just a default setting, and I believe I have tried that but something else that I can’t remember went wrong and I had to revert it, I will try again and update this post if I can get it to work.
Sorry, I have a pretty flaky memory, I should have written this sooner, but I hope this helps someone else out there somehow!
Update - 2024/06/07
Today, I finally decided to add the autocmd with some restrictions that I’ll explain below, here’s what that looked like:
-- file: lua/utils.lua
function M.handle_eol()
local eol = vim.api.nvim_buf_get_option(0, "eol")
local fixeol = vim.api.nvim_buf_get_option(0, "fixeol")
local current_filetype = vim.bo.filetype
local current_file_format = vim.bo.fileformat
if eol or fixeol and (current_file_format ~= "unix" or current_filetype == "php") then
M.set_eol({})
vim.notify("Handled EOL for this buffer", vim.log.levels.INFO)
end
end
function M.set_eol(opts)
vim.api.nvim_buf_set_option(0, "eol", false)
vim.api.nvim_buf_set_option(0, "fixeol", false)
vim.cmd([[ update ]])
if opts["save"] ~= nil and opts["save"] == true then
vim.cmd([[ w ]])
end
end
-- file: init.lua
vim.api.nvim_create_autocmd({ "BufReadPost", "BufNewFile" }, {
desc = "Handle EOL automatically depending on previous setting",
callback = utils.handle_eol,
})
The weird restrictions exist instead of just letting it do it for all files by default because I have tried that and it caused issues with certain files that I had to manually set eol for (they somehow had different settings and opening them caused them to create a diff I did not want). I also restricted the autocmd to PHP files because that is what I edit often, for other files, I added a keymap like this for doing that manually:
-- file: init.lua
local function set_eol()
utils.set_eol({ save = true })
end
vim.keymap.set("n", "<leader>eo", set_eol, { desc = "Disable and fix DOS EOL manually" })
These things aren’t really permanent solutions, I am open to whatever suggestions you might have (help!!) and I would love to hear from you!
Update - 2024/06/11
After having to deal with it again, it hit me that perhaps I was looking at the problem from the wrong perspective, maybe I had the wrong solution because I thought I had the right problem but didn’t. I finally decided stop being a lazy twat and took the time to investigate because I just knew the solutions I had tried weren’t the best and they certainly were not right. So, I launched PHPStorm to have a look at the current configuration, it was set to CLRF which is correct, I looked at a couple of other settings and then proceeded to do the one thing I should have taken 5 minutes to actually do; I read the f*cking manual (for Neovim).
I had read up on some of the EOL handling bits earlier but I didn’t really go far into the other bits because I just needed to have it fixed and get back to work. I finally read the Neovim docs for eol and fixeol with more attention and oh… It was right there, it finally clicked. I knew Vim’s behaviour and I knew how the line endings worked, I just didn’t put it together until now.
See, PHPStorm knew that the line had actually ended and did not need to add the carriage return or a new line but Vim saw that as unexpected (because of some “weird” EOL and EOF handling, this may be a better explanation of Vi’s - and in turn Vim’s - behaviour), so, it did what it knew how to, it “fixed” it by adding the carriage return on every line that did not have it and that was the last time there. Now that I finally knew the problem, the solution was rather simple, set nofixeol to false globally so it doesn’t fix it anymore, essentially telling Neovim; “Don’t fix my broken line endings, leave it as it is”.
vim.opt.fixeol = false
Or directly via Neovim’s command mode (or .vimrc):
:set nofixeol
So far, I have not had any issues with this and it has been working as expected, I will update this post if I run into any issues with this solution; which I don’t expect to happen.
This article is about a similar problem but the author WANTED a new line at the end of the file instead of removing the carriage return character.
TLDR;
- Windows uses
CRLFfor new lines, Linux usesLF - If you see
No newline at end of filein your diffs, it is likely aCRLFissue - You can often temporarily fix it in Vim for that session by running
:set noeol nofixeoland saving the file. - RTFM